-- Logs begin at Mon 2022-08-29 19:24:47 UTC, end at Mon 2022-08-29 20:53:19 UTC. -- Aug 29 20:04:16.569236 dv-cinder-852009-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 29 20:04:16.578273 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 29 20:04:16.580218 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 29 20:04:16 2022] *** Aug 29 20:04:16.580218 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 29 20:04:16.580218 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 29 20:04:16.580218 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: nodename: dv-cinder-852009-5-iscsi Aug 29 20:04:16.580218 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: machine: x86_64 Aug 29 20:04:16.580218 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: clock source: unix Aug 29 20:04:16.580218 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: pcre jit disabled Aug 29 20:04:16.580218 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: detected number of CPU cores: 4 Aug 29 20:04:16.580218 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: current working directory: / Aug 29 20:04:16.580218 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: detected binary path: /usr/bin/uwsgi-core Aug 29 20:04:16.580218 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: your processes number limit is 47680 Aug 29 20:04:16.580218 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: your memory page size is 4096 bytes Aug 29 20:04:16.580964 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: detected max file descriptor number: 2048 Aug 29 20:04:16.580964 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: lock engine: pthread robust mutexes Aug 29 20:04:16.580964 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: thunder lock: enabled Aug 29 20:04:16.580964 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 29 20:04:16.580964 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 29 20:04:16.620129 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: Python main interpreter initialized at 0x5639f43659c0 Aug 29 20:04:16.620129 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: python threads support enabled Aug 29 20:04:16.620129 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: your server socket listen backlog is limited to 100 connections Aug 29 20:04:16.620129 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: your mercy for graceful operations on workers is 90 seconds Aug 29 20:04:16.620517 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: mapped 403077 bytes (393 KB) for 2 cores Aug 29 20:04:16.620595 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: *** Operational MODE: preforking *** Aug 29 20:04:16.620880 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: *** uWSGI is running in multiple interpreter mode *** Aug 29 20:04:16.620880 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: spawned uWSGI master process (pid: 81846) Aug 29 20:04:16.621086 dv-cinder-852009-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 29 20:04:16.621877 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: spawned uWSGI worker 1 (pid: 81858, cores: 1) Aug 29 20:04:16.621981 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: spawned uWSGI worker 2 (pid: 81859, cores: 1) Aug 29 20:04:16.622249 dv-cinder-852009-5-iscsi devstack@keystone.service[81846]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 29 20:04:20.985600 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 20:04:20.985600 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: from cryptography.utils import int_from_bytes Aug 29 20:04:20.986518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 20:04:20.986518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: from cryptography.utils import int_from_bytes Aug 29 20:04:20.987408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 20:04:20.987408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: from cryptography.utils import int_from_bytes Aug 29 20:04:20.987883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 20:04:20.987883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: from cryptography.utils import int_from_bytes Aug 29 20:04:21.036896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 20:04:21.037090 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 20:04:21.037257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] command line args: [] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 20:04:21.037525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 20:04:21.037704 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ================================================================================ {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 20:04:21.037959 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] admin_token = **** {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.038209 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] config_dir = [] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.038288 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 20:04:21.038400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.038478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 20:04:21.038585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] config_source = [] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.038664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] command line args: [] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 20:04:21.038767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] debug = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.038919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 20:04:21.039003 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.039206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ================================================================================ {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 20:04:21.039332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.039443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] admin_token = **** {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.039546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.039641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] config_dir = [] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.039739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.039842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.039936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] config_source = [] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.040042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.040145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] debug = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.040284 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] list_limit = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.040389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.040522 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] log_config_append = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.040639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.040740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.040834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.040926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] log_dir = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.040926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] log_file = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.041101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.041207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.041308 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.041408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.041517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] list_limit = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.041635 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] log_config_append = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.041709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.041831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.041912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.041992 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] log_dir = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.042112 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.042188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] log_file = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.042302 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.042412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.042490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.042609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.042683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.042796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.042878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.042992 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.043072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.043191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.043271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.043390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.043473 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.043573 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.043724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.043847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.043972 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.044049 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.044269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.044400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.044522 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.044743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.044857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.85/identity {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.044980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] publish_errors = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.045098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.045215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.045327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.045488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.045602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.045713 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.045831 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.045970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.85/identity {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.046097 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.046215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] publish_errors = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.046333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] standard_threads = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.046451 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.046584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.046704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.046822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.046934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.047050 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.047161 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.047280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] use_journal = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.047392 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.047509 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] use_json = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.047509 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] use_stderr = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.047725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] standard_threads = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.047835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] use_syslog = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.047954 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.048072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.048214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.048336 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.048455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.048569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.048759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] use_journal = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.048759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] use_json = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.049044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.049160 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] use_stderr = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.049271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.049381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] use_syslog = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.049505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.049620 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:04:21.049738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.049849 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.049966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.050077 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.050198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.050311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.050423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.050533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.050644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.050754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.050873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.050986 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.051097 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.051213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.051325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.051436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.051555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.051675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.051788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.051905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.052025 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.052148 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.052293 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.052414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.052532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.052650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.052769 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.052893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.053006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.053118 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.053229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.053340 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.053453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.053573 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.053686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.053798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.053916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.054028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.054139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.054325 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.054446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.054565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.054682 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.054794 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.054913 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.055033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.055152 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.055271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.055384 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.055501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.055620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.055741 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.055858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.055969 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.056081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.056215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.056331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.056453 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.056566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.056677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.056794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.056905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.057018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.057136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.057249 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.057360 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.057470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.057581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.057675 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.057775 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.057870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.057967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.058063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.058193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.058302 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.058410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.058506 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.058599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.058693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.058799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.058912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.059031 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.059150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.059253 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.059375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.059488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.059613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.059736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.059850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.059971 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.060087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.060240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.060356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.060459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.060582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.060704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.060807 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.060925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.061039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.061144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.061264 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.061385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.061489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.061600 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.061718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.061822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.061940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.062061 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.062178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.062290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.062402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.062525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.062638 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.062750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.062862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.062982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.063097 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.063209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.063329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.063450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.063563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.063675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.063787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.063905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.064026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.064140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.064269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.064395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.064509 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.064609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.064680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.064741 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.064816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.064876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.064935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.064993 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.065131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.065256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.065373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.065485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.065603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.065720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.065841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.065961 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.066074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.066186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.066297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.066410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.066534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.066661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.066775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.066893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.067551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.067673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.067786 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.067907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.068021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.068133 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.068287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.068400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.068512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.068635 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.068750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.068862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.068974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.069084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.069201 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.069323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.069442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.069574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.069688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.069800 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.069910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.070030 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.070150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.070263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.070374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.070374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.070589 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.070709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.070823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.070934 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.071044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.071162 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.071274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.071394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.071512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.071618 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.071728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.071838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.071949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.072068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.072196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.072305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.072399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.072493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.072586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.072692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.072796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.072796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.072970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.073063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.073155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.073155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] auth.external = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.073348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.073481 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.074378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.074378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.074378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.074378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.074378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.074962 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] auth.external = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.075077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.075194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.075307 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.075421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.075535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] auth.password = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.075665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.075781 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] auth.token = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.075896 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] auth.password = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.076009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.076123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] auth.token = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.076255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.076386 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.076510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.076626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.076739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.076862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.076983 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.077107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.077222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.077336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.077450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.077564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.077676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] credential.caching = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.077798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.077915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.078029 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] credential.caching = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.078150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.078266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.078379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.078502 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.078617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.078731 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.078843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.078957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.079078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.079193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.079307 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.079428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.079543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.079656 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.079841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.079971 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.080094 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.080222 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.080338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.080452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.080578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.080695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.080809 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.080922 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.081036 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.081168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.081291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.081413 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.081527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] federation.caching = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.081527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.081747 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.081859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.081980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] federation.caching = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.081980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.082194 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.082194 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.082405 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.082517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.082678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.083330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.083449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.083563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.083676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.083790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.083912 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.084027 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.084148 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.084312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.084426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.084539 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.084666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.084783 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.caching = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.084897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.085010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.085124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.085236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.085359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.caching = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.085514 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.085633 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.085755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.085871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.085985 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.086107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.086226 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.086342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.086462 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.086645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.086762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.086885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.087001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.087115 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.087227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.087341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.087454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.087576 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.087691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.087811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.087926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.088040 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.088153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.088294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.088412 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.088525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.088639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.088752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.088865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.088989 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.089105 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.089219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.089333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.089447 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.089605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.089716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.089829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.089942 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.090054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.090167 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.090288 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.090403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.090518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.090631 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.090743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.090856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.090981 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.091097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.091210 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.091323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.091437 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.091551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.091672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.091778 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.091890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.092003 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.092116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.092275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.092402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.092518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.092633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.092746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.092859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.092972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.093093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.093209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.093323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.093436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.093549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.093672 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.093798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.093914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.094027 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.094141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.094254 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.094368 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.094489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.094605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.094719 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.094833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.094945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.095058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.095179 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.095294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.095408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.095521 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.095634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.095748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.095869 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.095985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.096102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.096230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.096345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.096458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.096583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.096698 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.096813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.096925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.097038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.097151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.097272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.097387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.097501 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.097620 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.097735 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.097849 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.097971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.098086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.098200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.098313 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.098425 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.098538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.098659 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.098659 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.098874 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.098986 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.099099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.099211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.099337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.099458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.099572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.099685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.099797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.099910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.100034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.100149 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.100284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.100397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.100510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.100623 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.100748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.100864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.100979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.101091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.101204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.101323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.101446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.101567 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.101682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.101795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.101907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.102029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.102144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.102257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.102370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.102483 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.102596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.102783 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.102901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.103015 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.103130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.103243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.103356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.103478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.103593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.103706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.103819 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.103933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.104046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.104237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.104356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.104469 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.104582 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.104695 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.104808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.104932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.105049 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.105162 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.105275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.105388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] resource.caching = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.105500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.105624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.105740 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.105853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.105966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.106082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.106194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] resource.caching = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.106316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.106432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.106546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.106658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.106772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.106884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.107005 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.107120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.107239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.107360 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.107475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.107587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.107708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] role.caching = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.107824 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.107943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] role.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.108057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.108188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.108304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.108429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.108545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] role.caching = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.108659 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.108773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] role.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.108885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.108998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.109120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.109235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.109348 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.109462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.109575 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.109688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.109809 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.109924 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.110037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.110037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.110248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.110362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.110484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.110599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.110713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.110826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.110938 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.111051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.111173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.111173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.111376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.111495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.111610 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.111715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.111811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.111906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.112000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.112000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.112235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.112363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.112480 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.112595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.112708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.112822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.112935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.113055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.113170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.113284 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.113397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.113510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.113625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.113745 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.113861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.113975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.114088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.114201 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.114314 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.114436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.114551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.114666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.114779 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.114892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.115005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.115127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.115242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.115363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.115478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.115591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] token.caching = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.115591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.115813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.115932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.116047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.116182 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.116299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.116299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.116522 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.116638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] token.caching = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.116753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.116866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.116979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.117092 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.117213 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.117327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.117441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.117553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.117667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.117667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.117886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.118000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.118114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.118227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.118339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.118451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.118572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.118687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.118801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.118914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.119035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.119158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.119273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.119273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.119495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.119608 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.119705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.connection = **** {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.119705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.119904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.119904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.120135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.120135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.120380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.120493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.120607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.120720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.120903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.connection = **** {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.121022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.121136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.121249 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.121363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.121475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.121596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.121712 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.121825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.121938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.122052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.122165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.122287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.122402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.122516 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.122629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.122742 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.122855 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.122976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.123091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.123206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.123319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.123432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.123545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.123670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.123785 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.123899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.124012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81858) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 20:04:21.124128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.124128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.124408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.124523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.124728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:04:21.124901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81859) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 20:04:21.154134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.154362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.154465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.154588 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.154673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.154819 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.162986 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.163249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.190795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.191027 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.191174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.191410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.245389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.245683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.295071 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.295299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.295394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.295566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.454253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.454441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.454558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.454636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.454709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.454809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.454886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.454959 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.455098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.455182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.455313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.455398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.478215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.478379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.478466 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.478618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.478702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.478845 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.478928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.479064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.479146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.479279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.479361 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.479489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.479566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.479703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.491198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.491343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.504077 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.504247 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.504388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.504529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.505135 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.505215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.505283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.505432 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.505528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.505651 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.505737 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.505866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.505950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81859) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.506082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81858) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:04:21.522787 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.522999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 20:04:21.523145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.523348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.523432 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 20:04:21.523551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 20:04:21.523688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.523826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.523909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 20:04:21.524093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.524266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.524386 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.524471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.524649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.524731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.524919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.535018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.535144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.535323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.535598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.535780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.536064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.536199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.536377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.543193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.543384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.546138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.546312 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.546425 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.546598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.546710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.546886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.546977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.547175 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.547256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.547465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.547554 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.547729 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.547819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.548004 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.548084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.548311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.548394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.548586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.548668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.548856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.548966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.549125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.572862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.574334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.574418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.574616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.576424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.576714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.579174 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.579467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.579756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.581560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.582034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.582525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.586589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.586883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.590872 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.591161 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.591447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.591730 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.592005 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.594211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.594504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.594601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.594795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.594923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.599028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.599330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.599619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.599909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.600219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.602416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.602706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.602993 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.610378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.616001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.616001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.618215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.618514 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.619053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.619355 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.622859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.623265 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.626080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.626389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.626689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.629215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.629580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.630379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.630666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.630962 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.631342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.631661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.634817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.637949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.638311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.638605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.638895 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.639336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.639669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.647243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.647532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.647804 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.648068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.655104 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.655876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.656192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.656460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.656745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.656840 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.659204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.659204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.661291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.661587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.661882 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.662166 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.662452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.663541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.665243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.666801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.667082 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.669775 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.670076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.670373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.670658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.670952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.673935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.674220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.674505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.674787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.675071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.675350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.675652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.682932 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.683260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.683549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.683837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.684125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.684429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.684725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.691711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.695330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.698236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.699686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.700725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.701197 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.701482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.701779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.703775 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.706707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.708213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.709045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.709701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.709989 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.710288 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.710526 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.713439 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.713743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.714033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.714310 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.716935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.718419 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.721416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.721634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.721927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.722209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.722307 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.722541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.722826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.730345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81858) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.730634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.730842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.730940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.731146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.731282 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.731420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.731557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.731708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.731967 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.732277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.732560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.732964 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.733471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.733967 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.734458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.734960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81858) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.738527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81859) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:04:21.738873 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.739666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.739944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.740242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.740792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.740792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.741064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.741351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.741640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.741914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.742179 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.742446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.742728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81859) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:04:21.772746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5639f43659c0 pid: 81858 (default app) Aug 29 20:04:21.776508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459bd3f-23c2-4748-a8d1-1efc391cfe42 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:21.776796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459bd3f-23c2-4748-a8d1-1efc391cfe42 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:21.777070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459bd3f-23c2-4748-a8d1-1efc391cfe42 None None] PATH_INFO: `/v3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:21.779146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5639f43659c0 pid: 81859 (default app) Aug 29 20:04:21.781230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1/1] 10.222.0.85 () {50 vars in 791 bytes} [Mon Aug 29 20:04:21 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:29.809027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3716ee8-99dd-4db0-b393-8b2095792b1d None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:29.809597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3716ee8-99dd-4db0-b393-8b2095792b1d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:29.809597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3716ee8-99dd-4db0-b393-8b2095792b1d None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:29.814187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1/2] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:04:29.819529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437effa-243e-4576-845c-edc5a04ec81f None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:29.820068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437effa-243e-4576-845c-edc5a04ec81f None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:29.820068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437effa-243e-4576-845c-edc5a04ec81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:29.889866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: INFO dbcounter [None req-b437effa-243e-4576-845c-edc5a04ec81f None None] Registered counter for database keystone Aug 29 20:04:29.912123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG oslo_db.sqlalchemy.engines [None req-b437effa-243e-4576-845c-edc5a04ec81f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81858) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 20:04:29.920871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writer thread running {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 20:04:30.105813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG passlib.handlers.bcrypt [None req-b437effa-243e-4576-845c-edc5a04ec81f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81858) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 20:04:30.106197 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG passlib.handlers.bcrypt [None req-b437effa-243e-4576-845c-edc5a04ec81f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81858) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 20:04:30.165480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b437effa-243e-4576-845c-edc5a04ec81f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:30.167096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b437effa-243e-4576-845c-edc5a04ec81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:30.298430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 2/3] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 20:04:30.769097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3716ee8-99dd-4db0-b393-8b2095792b1d None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:30.839493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: INFO dbcounter [None req-b3716ee8-99dd-4db0-b393-8b2095792b1d None None] Registered counter for database keystone Aug 29 20:04:30.858234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG oslo_db.sqlalchemy.engines [None req-b3716ee8-99dd-4db0-b393-8b2095792b1d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81859) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 20:04:30.864474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writer thread running {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 20:04:30.982433 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3716ee8-99dd-4db0-b393-8b2095792b1d None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:30.986755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603fac7-e036-4e90-926e-50c476fcf5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:30.987389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603fac7-e036-4e90-926e-50c476fcf5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:30.987673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603fac7-e036-4e90-926e-50c476fcf5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:30.987943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603fac7-e036-4e90-926e-50c476fcf5b2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:30.996591 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603fac7-e036-4e90-926e-50c476fcf5b2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:31.049193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7603fac7-e036-4e90-926e-50c476fcf5b2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 20:04:31.049536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603fac7-e036-4e90-926e-50c476fcf5b2 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:31.059512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 2/4] 10.222.0.85 () {58 vars in 1324 bytes} [Mon Aug 29 20:04:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:04:31.065547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437effa-243e-4576-845c-edc5a04ec81f None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:31.080252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437effa-243e-4576-845c-edc5a04ec81f None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:31.090065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf7d36-fc11-4a9d-a6a9-63e005fb3a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:31.091196 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf7d36-fc11-4a9d-a6a9-63e005fb3a21 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:31.091483 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf7d36-fc11-4a9d-a6a9-63e005fb3a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:31.091760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf7d36-fc11-4a9d-a6a9-63e005fb3a21 admin admin] PATH_INFO: `/v3/projects/e08700a8f1f34f3488594693a51c7bb4` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:31.093348 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf7d36-fc11-4a9d-a6a9-63e005fb3a21 admin admin] RBAC: Authorizing `identity:get_project(project_id=e08700a8f1f34f3488594693a51c7bb4)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:31.146369 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: WARNING keystone.common.rbac_enforcer.enforcer [None req-15cf7d36-fc11-4a9d-a6a9-63e005fb3a21 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 20:04:31.146656 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf7d36-fc11-4a9d-a6a9-63e005fb3a21 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:31.152244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 3/5] 10.222.0.85 () {56 vars in 1207 bytes} [Mon Aug 29 20:04:31 2022] GET /identity/v3/projects/e08700a8f1f34f3488594693a51c7bb4 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:33.076111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceafb91-9888-45ea-85f9-1cc3c9780160 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:33.076693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceafb91-9888-45ea-85f9-1cc3c9780160 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:33.076693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceafb91-9888-45ea-85f9-1cc3c9780160 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:33.080927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 3/6] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:33.086900 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fa6e8-8b10-4d03-bb97-995cd0052aa5 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:33.087369 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fa6e8-8b10-4d03-bb97-995cd0052aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:33.087472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fa6e8-8b10-4d03-bb97-995cd0052aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:33.108583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-080fa6e8-8b10-4d03-bb97-995cd0052aa5 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:33.110104 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-080fa6e8-8b10-4d03-bb97-995cd0052aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:33.118046 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 4/7] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 20:04:33.390641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceafb91-9888-45ea-85f9-1cc3c9780160 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:33.403497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceafb91-9888-45ea-85f9-1cc3c9780160 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:33.407711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867b0ec-83f5-4433-9450-9e41e356887c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:33.408348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867b0ec-83f5-4433-9450-9e41e356887c admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:33.408626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867b0ec-83f5-4433-9450-9e41e356887c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:33.408896 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867b0ec-83f5-4433-9450-9e41e356887c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:33.417272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867b0ec-83f5-4433-9450-9e41e356887c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:33.419058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867b0ec-83f5-4433-9450-9e41e356887c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:33.428303 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 4/8] 10.222.0.85 () {58 vars in 1324 bytes} [Mon Aug 29 20:04:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:04:33.433839 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fa6e8-8b10-4d03-bb97-995cd0052aa5 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:33.443371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fa6e8-8b10-4d03-bb97-995cd0052aa5 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:33.447680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e1ee8-998b-4f03-afc9-7bb2dabb3b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:33.448334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e1ee8-998b-4f03-afc9-7bb2dabb3b19 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:33.448620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e1ee8-998b-4f03-afc9-7bb2dabb3b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:33.448909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e1ee8-998b-4f03-afc9-7bb2dabb3b19 admin admin] PATH_INFO: `/v3/users/7afd1ca5c60c424cb58374006df79ec1` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:33.450241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e1ee8-998b-4f03-afc9-7bb2dabb3b19 admin admin] RBAC: Authorizing `identity:get_user(user_id=7afd1ca5c60c424cb58374006df79ec1)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:33.452056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e1ee8-998b-4f03-afc9-7bb2dabb3b19 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:33.453608 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 5/9] 10.222.0.85 () {56 vars in 1198 bytes} [Mon Aug 29 20:04:33 2022] GET /identity/v3/users/7afd1ca5c60c424cb58374006df79ec1 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:35.446528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab67bf-fb50-4193-82b1-4d5565e59c87 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:35.447045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab67bf-fb50-4193-82b1-4d5565e59c87 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:35.447045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab67bf-fb50-4193-82b1-4d5565e59c87 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:35.448401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 5/10] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:04:35.453723 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b936c-cddc-4dea-9ff0-2ddedabef46f None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:35.454116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b936c-cddc-4dea-9ff0-2ddedabef46f None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:35.454292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b936c-cddc-4dea-9ff0-2ddedabef46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:35.472893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-095b936c-cddc-4dea-9ff0-2ddedabef46f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:35.474124 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-095b936c-cddc-4dea-9ff0-2ddedabef46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:35.483650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc205c2-407d-4cf3-8caf-c37bd89a34f4 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:35.484038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc205c2-407d-4cf3-8caf-c37bd89a34f4 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:35.484351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc205c2-407d-4cf3-8caf-c37bd89a34f4 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:35.485701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 6/11] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:04:35.490618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8de2c-6abb-4efa-9cf5-abe81a400b3b None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:35.490903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8de2c-6abb-4efa-9cf5-abe81a400b3b None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:35.491187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8de2c-6abb-4efa-9cf5-abe81a400b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:35.533326 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG passlib.handlers.bcrypt [None req-7fb8de2c-6abb-4efa-9cf5-abe81a400b3b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81859) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 20:04:35.533892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG passlib.handlers.bcrypt [None req-7fb8de2c-6abb-4efa-9cf5-abe81a400b3b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81859) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 20:04:35.554816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 6/12] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:35.583689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7fb8de2c-6abb-4efa-9cf5-abe81a400b3b None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:35.585324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7fb8de2c-6abb-4efa-9cf5-abe81a400b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:35.687783 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 7/13] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:04:35.824938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b936c-cddc-4dea-9ff0-2ddedabef46f None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:35.848045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:35.935722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b936c-cddc-4dea-9ff0-2ddedabef46f None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:35.957299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8de2c-6abb-4efa-9cf5-abe81a400b3b None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:35.970369 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41992245-de96-434f-ad7e-2ac7ee09b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:35.971013 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41992245-de96-434f-ad7e-2ac7ee09b93f None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:35.971299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41992245-de96-434f-ad7e-2ac7ee09b93f None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:35.971582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41992245-de96-434f-ad7e-2ac7ee09b93f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:35.983293 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41992245-de96-434f-ad7e-2ac7ee09b93f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-41992245-de96-434f-ad7e-2ac7ee09b93f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:04:35.992772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:35.993871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:35.994699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:35.994699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:35.994699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:35.994699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:35.994699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:04:35.994699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:04:35.994699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:04:35.994699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:04:35.994699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:04:35.994699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:04:35.994699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:04:35.994699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:04:35.994699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 7/14] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:04:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:04:35.999789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41992245-de96-434f-ad7e-2ac7ee09b93f None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:36.065497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8de2c-6abb-4efa-9cf5-abe81a400b3b None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:36.069419 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:36.083847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:36.103193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41992245-de96-434f-ad7e-2ac7ee09b93f None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:36.130481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610410a-8498-4b3f-8a8a-42ffdbcb9051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:36.131987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610410a-8498-4b3f-8a8a-42ffdbcb9051 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:36.132526 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610410a-8498-4b3f-8a8a-42ffdbcb9051 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:36.133033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610410a-8498-4b3f-8a8a-42ffdbcb9051 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:36.137593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d7e5b-dd5c-4ade-a5b3-932cab566386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:36.138202 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d7e5b-dd5c-4ade-a5b3-932cab566386 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:36.138489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d7e5b-dd5c-4ade-a5b3-932cab566386 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:36.138770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d7e5b-dd5c-4ade-a5b3-932cab566386 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:36.139446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d7e5b-dd5c-4ade-a5b3-932cab566386 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:36.139754 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d7e5b-dd5c-4ade-a5b3-932cab566386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:36.141315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d7e5b-dd5c-4ade-a5b3-932cab566386 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:36.153966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 8/15] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:04:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:04:36.159427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d7e5b-dd5c-4ade-a5b3-932cab566386 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:36.213527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610410a-8498-4b3f-8a8a-42ffdbcb9051 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:36.215114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610410a-8498-4b3f-8a8a-42ffdbcb9051 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:36.258402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d7e5b-dd5c-4ade-a5b3-932cab566386 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:36.303256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8e22a-aadb-466a-bb51-9a53f6175a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:36.303887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8e22a-aadb-466a-bb51-9a53f6175a45 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:36.304198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8e22a-aadb-466a-bb51-9a53f6175a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:36.304471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8e22a-aadb-466a-bb51-9a53f6175a45 None admin] PATH_INFO: `/v3/users/7afd1ca5c60c424cb58374006df79ec1` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:36.305611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8e22a-aadb-466a-bb51-9a53f6175a45 None admin] RBAC: Authorizing `identity:get_user(user_id=7afd1ca5c60c424cb58374006df79ec1)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:36.307243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8e22a-aadb-466a-bb51-9a53f6175a45 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:36.308835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 9/16] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:04:36 2022] GET /identity/v3/users/7afd1ca5c60c424cb58374006df79ec1 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:04:36.312808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 8/17] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:04:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:04:36.317421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8e22a-aadb-466a-bb51-9a53f6175a45 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:36.317989 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610410a-8498-4b3f-8a8a-42ffdbcb9051 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:36.321949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:36.332500 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:36.418627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8e22a-aadb-466a-bb51-9a53f6175a45 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:36.422912 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610410a-8498-4b3f-8a8a-42ffdbcb9051 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:36.454150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ae09c-f973-44ab-96c8-207196743804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:36.454782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ae09c-f973-44ab-96c8-207196743804 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:36.455043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ae09c-f973-44ab-96c8-207196743804 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:36.455312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ae09c-f973-44ab-96c8-207196743804 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:36.456427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ae09c-f973-44ab-96c8-207196743804 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:36.457036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7310-7e6e-4a8b-a095-513ecfd12cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:36.457887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7310-7e6e-4a8b-a095-513ecfd12cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:36.457983 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ae09c-f973-44ab-96c8-207196743804 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:36.458168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7310-7e6e-4a8b-a095-513ecfd12cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:36.458439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7310-7e6e-4a8b-a095-513ecfd12cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:36.459479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 10/18] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:04:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:36.465282 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ae09c-f973-44ab-96c8-207196743804 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:36.476726 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f7310-7e6e-4a8b-a095-513ecfd12cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:36.478226 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f7310-7e6e-4a8b-a095-513ecfd12cd6 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-fe3f7310-7e6e-4a8b-a095-513ecfd12cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:04:36.496979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:36.498051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:04:36.499194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:04:36.499194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:04:36.499194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:04:36.499194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 9/19] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:04:36.506892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7310-7e6e-4a8b-a095-513ecfd12cd6 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:36.557977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:36.562816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:36.583652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ae09c-f973-44ab-96c8-207196743804 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:36.607609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7310-7e6e-4a8b-a095-513ecfd12cd6 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:36.620697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56dacf-b633-49bb-9e92-a44e89abd5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:36.621306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56dacf-b633-49bb-9e92-a44e89abd5fc None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:36.621586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56dacf-b633-49bb-9e92-a44e89abd5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:36.621859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56dacf-b633-49bb-9e92-a44e89abd5fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:36.622581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56dacf-b633-49bb-9e92-a44e89abd5fc None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.domain.id=default, user.id=7afd1ca5c60c424cb58374006df79ec1) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:36.622888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56dacf-b633-49bb-9e92-a44e89abd5fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:36.624342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56dacf-b633-49bb-9e92-a44e89abd5fc None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:36.632324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 11/20] 10.222.0.85 () {56 vars in 1322 bytes} [Mon Aug 29 20:04:36 2022] GET /identity/v3/role_assignments?user.id=7afd1ca5c60c424cb58374006df79ec1&scope.domain.id=default&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 233 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:04:36.643680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4c302-5372-4bcf-b3d7-bfcc50c94c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:36.644369 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4c302-5372-4bcf-b3d7-bfcc50c94c3f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:36.644652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4c302-5372-4bcf-b3d7-bfcc50c94c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:36.644923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4c302-5372-4bcf-b3d7-bfcc50c94c3f None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:36.645583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4c302-5372-4bcf-b3d7-bfcc50c94c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:36.645904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4c302-5372-4bcf-b3d7-bfcc50c94c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:36.647346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4c302-5372-4bcf-b3d7-bfcc50c94c3f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:36.667132 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 10/21] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:04:38.654078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c2126-a123-4a7d-b642-be78ec4cb26b None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:38.654615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c2126-a123-4a7d-b642-be78ec4cb26b None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:38.654615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c2126-a123-4a7d-b642-be78ec4cb26b None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:38.656280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 12/22] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:38.663569 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a764f-9818-4319-a610-353655a856d4 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:38.664055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a764f-9818-4319-a610-353655a856d4 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:38.664127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a764f-9818-4319-a610-353655a856d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:38.701343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-239a764f-9818-4319-a610-353655a856d4 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:38.702578 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-239a764f-9818-4319-a610-353655a856d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:38.794254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 11/23] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:04:38.931281 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5347a28-032a-4158-b271-648b65764aea None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:38.932374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5347a28-032a-4158-b271-648b65764aea None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:38.933214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5347a28-032a-4158-b271-648b65764aea None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:38.936039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 13/24] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:38.958138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39827a3f-df12-4fca-abfd-1c020ba689d6 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:38.959086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39827a3f-df12-4fca-abfd-1c020ba689d6 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:38.959843 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39827a3f-df12-4fca-abfd-1c020ba689d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:38.995667 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-39827a3f-df12-4fca-abfd-1c020ba689d6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:38.997509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-39827a3f-df12-4fca-abfd-1c020ba689d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:39.058875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:39.086474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 12/25] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:39.199751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5347a28-032a-4158-b271-648b65764aea None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:39.348860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5347a28-032a-4158-b271-648b65764aea None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:39.378109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12780e-7232-4eed-8d13-820bdfc42f73 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:39.378673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12780e-7232-4eed-8d13-820bdfc42f73 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:39.379101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12780e-7232-4eed-8d13-820bdfc42f73 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:39.382545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 13/26] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:04:39.388265 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835c6f9-65b1-4093-8101-2c1fa2782e37 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:39.388751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835c6f9-65b1-4093-8101-2c1fa2782e37 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:39.389173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835c6f9-65b1-4093-8101-2c1fa2782e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:39.401187 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8bc74-d503-43ec-a953-b19e1b880a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:39.401829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8bc74-d503-43ec-a953-b19e1b880a09 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:39.402115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8bc74-d503-43ec-a953-b19e1b880a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:39.402392 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8bc74-d503-43ec-a953-b19e1b880a09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:39.421551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f835c6f9-65b1-4093-8101-2c1fa2782e37 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:39.423028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f835c6f9-65b1-4093-8101-2c1fa2782e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:39.425052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8bc74-d503-43ec-a953-b19e1b880a09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-c4c8bc74-d503-43ec-a953-b19e1b880a09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:04:39.432000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:39.433227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:39.434143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:39.434143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:39.434143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:39.434143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:39.434143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:04:39.434143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:04:39.434143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:04:39.434143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:04:39.434143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:04:39.434143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:04:39.434143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:04:39.434143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:04:39.434143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 14/27] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:04:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:04:39.440323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8bc74-d503-43ec-a953-b19e1b880a09 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:39.447206 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:39.530618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 14/28] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:39.622249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03fb52-10dc-40d2-ab80-1352c78509ae None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:39.622832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03fb52-10dc-40d2-ab80-1352c78509ae None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:39.623304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03fb52-10dc-40d2-ab80-1352c78509ae None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:39.623403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8bc74-d503-43ec-a953-b19e1b880a09 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:39.625141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 15/29] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:39.636899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aa375-c6b8-4050-8105-40faea52ac47 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:39.637499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aa375-c6b8-4050-8105-40faea52ac47 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:39.637979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aa375-c6b8-4050-8105-40faea52ac47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:39.677386 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-af9aa375-c6b8-4050-8105-40faea52ac47 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:39.680467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-af9aa375-c6b8-4050-8105-40faea52ac47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:39.695056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edaebfe-c18e-4dc1-aa15-8607b0b57a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:39.695721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edaebfe-c18e-4dc1-aa15-8607b0b57a4e None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:39.695959 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edaebfe-c18e-4dc1-aa15-8607b0b57a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:39.696262 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edaebfe-c18e-4dc1-aa15-8607b0b57a4e None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:39.696946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edaebfe-c18e-4dc1-aa15-8607b0b57a4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:39.697252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edaebfe-c18e-4dc1-aa15-8607b0b57a4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:39.698723 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edaebfe-c18e-4dc1-aa15-8607b0b57a4e None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:39.712677 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 15/30] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:04:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:04:39.713513 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edaebfe-c18e-4dc1-aa15-8607b0b57a4e None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:39.805756 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 16/31] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:04:39.806560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9aa375-c6b8-4050-8105-40faea52ac47 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:39.826404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:39.866101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edaebfe-c18e-4dc1-aa15-8607b0b57a4e None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:39.913623 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a8eba-c588-4df9-a70c-98debd0426c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:39.914265 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a8eba-c588-4df9-a70c-98debd0426c8 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:39.914552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a8eba-c588-4df9-a70c-98debd0426c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:39.914838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a8eba-c588-4df9-a70c-98debd0426c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:39.915320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:39.933920 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a8eba-c588-4df9-a70c-98debd0426c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:39.938081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a8eba-c588-4df9-a70c-98debd0426c8 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-2d0a8eba-c588-4df9-a70c-98debd0426c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:04:39.968017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:39.968935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:04:39.970236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:04:39.970236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:04:39.970236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:04:39.970236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 16/32] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:04:39.970236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a8eba-c588-4df9-a70c-98debd0426c8 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:40.031146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9aa375-c6b8-4050-8105-40faea52ac47 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:40.090870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adcd8b-94c6-4a77-b06e-e6cb9bba1d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:40.091444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adcd8b-94c6-4a77-b06e-e6cb9bba1d1a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:40.091730 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adcd8b-94c6-4a77-b06e-e6cb9bba1d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:40.092009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adcd8b-94c6-4a77-b06e-e6cb9bba1d1a None admin] PATH_INFO: `/v3/users/7afd1ca5c60c424cb58374006df79ec1` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:40.093538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adcd8b-94c6-4a77-b06e-e6cb9bba1d1a None admin] RBAC: Authorizing `identity:get_user(user_id=7afd1ca5c60c424cb58374006df79ec1)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:40.095126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adcd8b-94c6-4a77-b06e-e6cb9bba1d1a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:40.097108 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 17/33] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:04:39 2022] GET /identity/v3/users/7afd1ca5c60c424cb58374006df79ec1 => generated 247 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:40.098330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adcd8b-94c6-4a77-b06e-e6cb9bba1d1a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:40.183228 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a8eba-c588-4df9-a70c-98debd0426c8 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:40.202948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:40.221512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e7bde-c937-4247-9789-e8d6cefd7aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:40.222222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e7bde-c937-4247-9789-e8d6cefd7aa6 None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:40.222642 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e7bde-c937-4247-9789-e8d6cefd7aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:40.223004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e7bde-c937-4247-9789-e8d6cefd7aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:40.223760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e7bde-c937-4247-9789-e8d6cefd7aa6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:40.225277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e7bde-c937-4247-9789-e8d6cefd7aa6 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:40.228402 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adcd8b-94c6-4a77-b06e-e6cb9bba1d1a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:40.230991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:40.266015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd84ed9-8219-460d-83b4-18f319bb7622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:40.266761 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd84ed9-8219-460d-83b4-18f319bb7622 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:40.267044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd84ed9-8219-460d-83b4-18f319bb7622 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:40.267322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd84ed9-8219-460d-83b4-18f319bb7622 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:40.267976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd84ed9-8219-460d-83b4-18f319bb7622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:40.268313 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd84ed9-8219-460d-83b4-18f319bb7622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:40.269746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd84ed9-8219-460d-83b4-18f319bb7622 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:40.281963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 18/34] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:40.287037 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd84ed9-8219-460d-83b4-18f319bb7622 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:40.408401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd84ed9-8219-460d-83b4-18f319bb7622 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:40.447828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f20e6e-552a-4f8d-b9d0-6c17496a7017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:40.448512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f20e6e-552a-4f8d-b9d0-6c17496a7017 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:40.448798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f20e6e-552a-4f8d-b9d0-6c17496a7017 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:40.449076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f20e6e-552a-4f8d-b9d0-6c17496a7017 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:40.450411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f20e6e-552a-4f8d-b9d0-6c17496a7017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:40.452155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f20e6e-552a-4f8d-b9d0-6c17496a7017 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:40.453826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 19/35] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:04:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:40.454568 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f20e6e-552a-4f8d-b9d0-6c17496a7017 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:40.480250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 17/36] 10.222.0.85 () {60 vars in 1131 bytes} [Mon Aug 29 20:04:39 2022] POST /identity/v3/roles => generated 219 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:04:40.481726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e7bde-c937-4247-9789-e8d6cefd7aa6 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:40.514740 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:40.620358 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e7bde-c937-4247-9789-e8d6cefd7aa6 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:40.635549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f20e6e-552a-4f8d-b9d0-6c17496a7017 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:40.659230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b6b62-37c4-4488-bcf8-5e086d61cbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:40.660630 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b6b62-37c4-4488-bcf8-5e086d61cbfe None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:40.660630 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b6b62-37c4-4488-bcf8-5e086d61cbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:40.660902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b6b62-37c4-4488-bcf8-5e086d61cbfe None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:40.661532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b6b62-37c4-4488-bcf8-5e086d61cbfe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:40.663164 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b6b62-37c4-4488-bcf8-5e086d61cbfe None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:40.678182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c042e-d548-4055-a26f-7c91eb1df3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:40.678875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c042e-d548-4055-a26f-7c91eb1df3f7 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:40.679085 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c042e-d548-4055-a26f-7c91eb1df3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:40.679364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c042e-d548-4055-a26f-7c91eb1df3f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:40.680668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c042e-d548-4055-a26f-7c91eb1df3f7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:40.681565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c042e-d548-4055-a26f-7c91eb1df3f7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:40.841333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 18/37] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:04:40 2022] POST /identity/v3/projects => generated 301 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:04:40.842146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b6b62-37c4-4488-bcf8-5e086d61cbfe None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:40.904918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:40.985461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b6b62-37c4-4488-bcf8-5e086d61cbfe None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:41.009269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 20/38] 10.222.0.85 () {60 vars in 1131 bytes} [Mon Aug 29 20:04:40 2022] POST /identity/v3/roles => generated 225 bytes in 555 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:04:41.010941 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab149d-4a8a-49d5-a0e9-f49ea0a40b38 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:41.011351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab149d-4a8a-49d5-a0e9-f49ea0a40b38 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:41.011787 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab149d-4a8a-49d5-a0e9-f49ea0a40b38 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:41.013190 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 21/39] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:41.023350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ebdcb-f456-431b-9e93-3f1536543ee0 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:41.023977 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ebdcb-f456-431b-9e93-3f1536543ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:41.024382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ebdcb-f456-431b-9e93-3f1536543ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:41.037031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb39ad0-91e9-4d87-bbbc-72178bc2112f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:41.038056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb39ad0-91e9-4d87-bbbc-72178bc2112f None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:41.038529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb39ad0-91e9-4d87-bbbc-72178bc2112f None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:41.039020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb39ad0-91e9-4d87-bbbc-72178bc2112f None admin] PATH_INFO: `/v3/domains/default/users/7afd1ca5c60c424cb58374006df79ec1/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:41.042717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb39ad0-91e9-4d87-bbbc-72178bc2112f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7afd1ca5c60c424cb58374006df79ec1, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:41.045075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb39ad0-91e9-4d87-bbbc-72178bc2112f None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:41.048320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-2b0ebdcb-f456-431b-9e93-3f1536543ee0 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:41.051006 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2b0ebdcb-f456-431b-9e93-3f1536543ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:41.133419 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 22/40] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:41.134869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b769a-018b-4742-8d24-5349c133caa9 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:41.135167 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b769a-018b-4742-8d24-5349c133caa9 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:41.135457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b769a-018b-4742-8d24-5349c133caa9 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:41.139731 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 23/41] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:41.142711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254a707-5f72-4177-a269-8b332054afe0 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:41.143001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254a707-5f72-4177-a269-8b332054afe0 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:41.143278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254a707-5f72-4177-a269-8b332054afe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:41.166761 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 19/42] 10.222.0.85 () {58 vars in 1361 bytes} [Mon Aug 29 20:04:40 2022] PUT /identity/v3/domains/default/users/7afd1ca5c60c424cb58374006df79ec1/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:04:41.169021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-3254a707-5f72-4177-a269-8b332054afe0 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:41.170609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3254a707-5f72-4177-a269-8b332054afe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:41.192854 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:41.255362 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 24/43] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:04:41.428151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb39ad0-91e9-4d87-bbbc-72178bc2112f None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:41.524581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254a707-5f72-4177-a269-8b332054afe0 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:41.538176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:41.547211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb39ad0-91e9-4d87-bbbc-72178bc2112f None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:41.585899 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d34cb-af83-4b1a-b082-3dbd2554cf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:41.587174 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d34cb-af83-4b1a-b082-3dbd2554cf8a None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:41.587609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d34cb-af83-4b1a-b082-3dbd2554cf8a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:41.588072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d34cb-af83-4b1a-b082-3dbd2554cf8a None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:41.588951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d34cb-af83-4b1a-b082-3dbd2554cf8a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:41.590573 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d34cb-af83-4b1a-b082-3dbd2554cf8a None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:41.639539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254a707-5f72-4177-a269-8b332054afe0 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:41.675673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90a681-4bcb-45a6-975a-1ee220387ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:41.676412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90a681-4bcb-45a6-975a-1ee220387ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:41.676698 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90a681-4bcb-45a6-975a-1ee220387ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:41.676975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90a681-4bcb-45a6-975a-1ee220387ad0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:41.678122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90a681-4bcb-45a6-975a-1ee220387ad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:41.679692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90a681-4bcb-45a6-975a-1ee220387ad0 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:41.681560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 25/44] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:04:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:04:41.686805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90a681-4bcb-45a6-975a-1ee220387ad0 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:41.688235 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 20/45] 10.222.0.85 () {60 vars in 1131 bytes} [Mon Aug 29 20:04:41 2022] POST /identity/v3/roles => generated 223 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:04:41.713314 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:41.802607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90a681-4bcb-45a6-975a-1ee220387ad0 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:41.842179 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33633fc1-7d15-46ac-89d3-93689c04fe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:41.842179 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33633fc1-7d15-46ac-89d3-93689c04fe41 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:41.842455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33633fc1-7d15-46ac-89d3-93689c04fe41 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:41.842678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33633fc1-7d15-46ac-89d3-93689c04fe41 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:41.843267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33633fc1-7d15-46ac-89d3-93689c04fe41 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:41.844897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33633fc1-7d15-46ac-89d3-93689c04fe41 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:41.932098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 26/46] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:04:41 2022] POST /identity/v3/projects => generated 312 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:04:43.110810 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa665dab-ac83-4e8b-ab95-09e2e6a6ec77 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:43.111392 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa665dab-ac83-4e8b-ab95-09e2e6a6ec77 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:43.111392 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa665dab-ac83-4e8b-ab95-09e2e6a6ec77 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:43.112724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 21/47] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:43.118461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a5f2b-97ab-4c82-99de-462ace3f48d4 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:43.118846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a5f2b-97ab-4c82-99de-462ace3f48d4 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:43.119143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a5f2b-97ab-4c82-99de-462ace3f48d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:43.137557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-797a5f2b-97ab-4c82-99de-462ace3f48d4 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:43.138756 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-797a5f2b-97ab-4c82-99de-462ace3f48d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:43.204531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 27/48] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:43.474904 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa665dab-ac83-4e8b-ab95-09e2e6a6ec77 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:43.684709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa665dab-ac83-4e8b-ab95-09e2e6a6ec77 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:43.731848 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49a253-eed4-4fdb-b66d-81032cbf170c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:43.732441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49a253-eed4-4fdb-b66d-81032cbf170c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:43.732725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49a253-eed4-4fdb-b66d-81032cbf170c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:43.733001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49a253-eed4-4fdb-b66d-81032cbf170c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:43.743685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49a253-eed4-4fdb-b66d-81032cbf170c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-fd49a253-eed4-4fdb-b66d-81032cbf170c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:04:43.748542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:43.749547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:43.750525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:43.750525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:43.750525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:43.750525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:43.750525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:04:43.750525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:04:43.750525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:04:43.750525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:04:43.750525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:04:43.750525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:04:43.750525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:04:43.750525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:04:43.750525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 22/49] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:04:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:04:43.755470 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a5f2b-97ab-4c82-99de-462ace3f48d4 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:43.757686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:43.854590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a5f2b-97ab-4c82-99de-462ace3f48d4 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:43.871339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a71ca-d4ac-4671-a86a-4a847a50ccec None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:43.871626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a71ca-d4ac-4671-a86a-4a847a50ccec None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:43.871905 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a71ca-d4ac-4671-a86a-4a847a50ccec None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:43.873423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 23/50] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:04:43.878208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdf0e1-2c0a-4ebf-a451-f1195f7e944c None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:43.878561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdf0e1-2c0a-4ebf-a451-f1195f7e944c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:43.879361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdf0e1-2c0a-4ebf-a451-f1195f7e944c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:43.889165 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7460a-a621-4b96-bb4e-99ed9e42eb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:43.889783 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7460a-a621-4b96-bb4e-99ed9e42eb62 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:43.890064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7460a-a621-4b96-bb4e-99ed9e42eb62 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:43.890339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7460a-a621-4b96-bb4e-99ed9e42eb62 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:43.891004 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7460a-a621-4b96-bb4e-99ed9e42eb62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:43.891310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7460a-a621-4b96-bb4e-99ed9e42eb62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:43.892760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7460a-a621-4b96-bb4e-99ed9e42eb62 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:43.897333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ebcdf0e1-2c0a-4ebf-a451-f1195f7e944c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:43.898869 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ebcdf0e1-2c0a-4ebf-a451-f1195f7e944c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:43.902497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:43.905406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 28/51] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:04:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:43.911088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7460a-a621-4b96-bb4e-99ed9e42eb62 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:43.970175 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 24/52] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:04:44.077338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:44.114274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7460a-a621-4b96-bb4e-99ed9e42eb62 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:44.147025 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb4615-0cae-44ec-b961-6f7691466fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:44.147662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb4615-0cae-44ec-b961-6f7691466fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:44.147946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb4615-0cae-44ec-b961-6f7691466fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:44.148258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb4615-0cae-44ec-b961-6f7691466fe3 None admin] PATH_INFO: `/v3/users/7afd1ca5c60c424cb58374006df79ec1` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:44.149412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb4615-0cae-44ec-b961-6f7691466fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=7afd1ca5c60c424cb58374006df79ec1)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:44.151116 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb4615-0cae-44ec-b961-6f7691466fe3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:44.152816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 29/53] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:04:43 2022] GET /identity/v3/users/7afd1ca5c60c424cb58374006df79ec1 => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:04:44.158769 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdf0e1-2c0a-4ebf-a451-f1195f7e944c None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:44.252142 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb4615-0cae-44ec-b961-6f7691466fe3 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:44.269683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdf0e1-2c0a-4ebf-a451-f1195f7e944c None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:44.304662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd3143-b9da-4817-b1c6-9e258e025a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:44.305326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd3143-b9da-4817-b1c6-9e258e025a05 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:44.305654 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd3143-b9da-4817-b1c6-9e258e025a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:44.305962 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd3143-b9da-4817-b1c6-9e258e025a05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:44.307191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd3143-b9da-4817-b1c6-9e258e025a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:44.308903 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd3143-b9da-4817-b1c6-9e258e025a05 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:44.310682 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 25/54] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:04:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:44.317186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd3143-b9da-4817-b1c6-9e258e025a05 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:44.328017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:44.359215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb4615-0cae-44ec-b961-6f7691466fe3 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:44.388036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:44.393236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a92d1-53cf-4793-b5b9-4d991bda8f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:44.393876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a92d1-53cf-4793-b5b9-4d991bda8f61 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:44.394754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a92d1-53cf-4793-b5b9-4d991bda8f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:44.395036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a92d1-53cf-4793-b5b9-4d991bda8f61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:44.396151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a92d1-53cf-4793-b5b9-4d991bda8f61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:44.397629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a92d1-53cf-4793-b5b9-4d991bda8f61 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:44.399233 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 30/55] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:04:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:04:44.405000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a92d1-53cf-4793-b5b9-4d991bda8f61 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:44.427448 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd3143-b9da-4817-b1c6-9e258e025a05 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:44.467457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d8df9-c53c-4555-8cab-113b9b5a5f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:44.468135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d8df9-c53c-4555-8cab-113b9b5a5f31 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:44.468443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d8df9-c53c-4555-8cab-113b9b5a5f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:44.468727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d8df9-c53c-4555-8cab-113b9b5a5f31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:44.469527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d8df9-c53c-4555-8cab-113b9b5a5f31 None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.domain.id=default, user.id=7afd1ca5c60c424cb58374006df79ec1) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:44.469917 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d8df9-c53c-4555-8cab-113b9b5a5f31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:44.471523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d8df9-c53c-4555-8cab-113b9b5a5f31 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:44.477615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 26/56] 10.222.0.85 () {56 vars in 1322 bytes} [Mon Aug 29 20:04:44 2022] GET /identity/v3/role_assignments?user.id=7afd1ca5c60c424cb58374006df79ec1&scope.domain.id=default&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 530 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:04:44.507965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a92d1-53cf-4793-b5b9-4d991bda8f61 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:44.549621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cd815-f139-4c21-8053-b15c5dce7eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:44.550393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cd815-f139-4c21-8053-b15c5dce7eb8 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:44.550683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cd815-f139-4c21-8053-b15c5dce7eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:44.550962 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cd815-f139-4c21-8053-b15c5dce7eb8 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:44.552046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cd815-f139-4c21-8053-b15c5dce7eb8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:44.554032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cd815-f139-4c21-8053-b15c5dce7eb8 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:44.658079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 31/57] 10.222.0.85 () {60 vars in 1140 bytes} [Mon Aug 29 20:04:44 2022] POST /identity/v3/projects => generated 298 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:04:46.555839 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e333e7e-d6ea-4413-8d9a-2d1636e47075 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:46.556419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e333e7e-d6ea-4413-8d9a-2d1636e47075 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:46.556419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e333e7e-d6ea-4413-8d9a-2d1636e47075 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:46.557759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 27/58] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:46.563085 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3155f0c-a68c-4527-976a-4f92883a99b4 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:46.563512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3155f0c-a68c-4527-976a-4f92883a99b4 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:46.563646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3155f0c-a68c-4527-976a-4f92883a99b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:46.582584 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-b3155f0c-a68c-4527-976a-4f92883a99b4 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:46.583831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b3155f0c-a68c-4527-976a-4f92883a99b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:46.630324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:46.652570 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 32/59] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:46.924711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e333e7e-d6ea-4413-8d9a-2d1636e47075 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:46.995483 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:47.026037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e333e7e-d6ea-4413-8d9a-2d1636e47075 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:47.057944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8adf4-eced-4253-bdfd-eef5a190b286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:47.058569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8adf4-eced-4253-bdfd-eef5a190b286 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:47.058853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8adf4-eced-4253-bdfd-eef5a190b286 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:47.059130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8adf4-eced-4253-bdfd-eef5a190b286 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:47.060255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8adf4-eced-4253-bdfd-eef5a190b286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:47.061700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8adf4-eced-4253-bdfd-eef5a190b286 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:47.063260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 28/60] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:04:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:47.072850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3155f0c-a68c-4527-976a-4f92883a99b4 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:47.168600 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3155f0c-a68c-4527-976a-4f92883a99b4 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:47.200785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31188445-264b-43fd-9e71-eb420c27a2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:47.201323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31188445-264b-43fd-9e71-eb420c27a2b3 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:47.201606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31188445-264b-43fd-9e71-eb420c27a2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:47.201882 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31188445-264b-43fd-9e71-eb420c27a2b3 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:47.202527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31188445-264b-43fd-9e71-eb420c27a2b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:47.203993 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31188445-264b-43fd-9e71-eb420c27a2b3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:47.462971 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 33/61] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:04:47 2022] POST /identity/v3/users => generated 275 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:04:50.406221 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7586a3-00e6-4691-9610-fba8b2015424 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:50.407267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7586a3-00e6-4691-9610-fba8b2015424 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:50.407759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7586a3-00e6-4691-9610-fba8b2015424 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:50.410134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 29/62] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:50.415304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48f644-ceea-4744-883d-f4c4e4382955 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:50.416209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48f644-ceea-4744-883d-f4c4e4382955 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:50.416693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48f644-ceea-4744-883d-f4c4e4382955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:50.442233 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-ca48f644-ceea-4744-883d-f4c4e4382955 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:50.444125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ca48f644-ceea-4744-883d-f4c4e4382955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:50.486290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:50.536566 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 34/63] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:50.759507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f5d51-c291-49f2-8eac-ab291e4b25b4 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:50.760158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f5d51-c291-49f2-8eac-ab291e4b25b4 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:50.760672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f5d51-c291-49f2-8eac-ab291e4b25b4 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:50.762313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 30/64] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:50.768094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf1dd9-ebe9-4dc5-8625-b55203a0c6a8 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:50.768625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf1dd9-ebe9-4dc5-8625-b55203a0c6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:50.769445 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf1dd9-ebe9-4dc5-8625-b55203a0c6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:50.796210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-8aaf1dd9-ebe9-4dc5-8625-b55203a0c6a8 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:50.797430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8aaf1dd9-ebe9-4dc5-8625-b55203a0c6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:50.886435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 35/65] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:50.926863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f5d51-c291-49f2-8eac-ab291e4b25b4 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:51.076452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f5d51-c291-49f2-8eac-ab291e4b25b4 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:51.086819 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107c457-2e72-4aff-a111-1b1910345035 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.087115 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107c457-2e72-4aff-a111-1b1910345035 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.087390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107c457-2e72-4aff-a111-1b1910345035 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.089408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 36/66] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:51.094545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d2458-3f76-41c5-8a82-7de0eae4e2e2 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.094545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d2458-3f76-41c5-8a82-7de0eae4e2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.094831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d2458-3f76-41c5-8a82-7de0eae4e2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.123588 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca9686-8b2d-4c95-875c-f41c57aba619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:51.124658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca9686-8b2d-4c95-875c-f41c57aba619 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.125146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca9686-8b2d-4c95-875c-f41c57aba619 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.125639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca9686-8b2d-4c95-875c-f41c57aba619 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.129522 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-827d2458-3f76-41c5-8a82-7de0eae4e2e2 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:51.130522 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-827d2458-3f76-41c5-8a82-7de0eae4e2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:51.135638 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:51.139195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca9686-8b2d-4c95-875c-f41c57aba619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-a0ca9686-8b2d-4c95-875c-f41c57aba619 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:04:51.144422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:51.151325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 31/67] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:04:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:04:51.152079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88adc7-475b-4f30-b7ee-84aa21ba761c None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.153312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88adc7-475b-4f30-b7ee-84aa21ba761c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.153312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88adc7-475b-4f30-b7ee-84aa21ba761c None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.153312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 32/68] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:51.153312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25291a4b-da9e-466a-a391-347a003f7a3c None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.153312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25291a4b-da9e-466a-a391-347a003f7a3c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.153312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25291a4b-da9e-466a-a391-347a003f7a3c None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.153312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 33/69] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:51.153768 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25291a4b-da9e-466a-a391-347a003f7a3c None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:51.202225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 37/70] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:04:51.204246 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b277a3c-4033-46b9-9c3f-2914b8fb0de1 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.204617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b277a3c-4033-46b9-9c3f-2914b8fb0de1 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.204933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b277a3c-4033-46b9-9c3f-2914b8fb0de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.234671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-4b277a3c-4033-46b9-9c3f-2914b8fb0de1 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:51.236794 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4b277a3c-4033-46b9-9c3f-2914b8fb0de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:51.269372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25291a4b-da9e-466a-a391-347a003f7a3c None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:51.269890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:51.312205 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65a321-c6e7-4340-aa2c-39cbbc50daf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:51.312932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65a321-c6e7-4340-aa2c-39cbbc50daf1 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.313316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65a321-c6e7-4340-aa2c-39cbbc50daf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.313604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65a321-c6e7-4340-aa2c-39cbbc50daf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.314273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65a321-c6e7-4340-aa2c-39cbbc50daf1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:51.314629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65a321-c6e7-4340-aa2c-39cbbc50daf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:51.322625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65a321-c6e7-4340-aa2c-39cbbc50daf1 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:51.337703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 34/71] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:04:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:51.338719 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94695003-dc34-481d-8bb0-b2732e4a6dc9 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.339000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94695003-dc34-481d-8bb0-b2732e4a6dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.339276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94695003-dc34-481d-8bb0-b2732e4a6dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.341188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 38/72] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:51.342395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b277a3c-4033-46b9-9c3f-2914b8fb0de1 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:51.364560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-94695003-dc34-481d-8bb0-b2732e4a6dc9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:51.365765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-94695003-dc34-481d-8bb0-b2732e4a6dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:51.465123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:51.481305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 35/73] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:51.481985 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94695003-dc34-481d-8bb0-b2732e4a6dc9 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:51.499985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b277a3c-4033-46b9-9c3f-2914b8fb0de1 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:51.552476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44232794-a266-4e1f-afdb-d4d72e55b6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:51.554526 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44232794-a266-4e1f-afdb-d4d72e55b6ab None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.554810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44232794-a266-4e1f-afdb-d4d72e55b6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.555305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44232794-a266-4e1f-afdb-d4d72e55b6ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.582739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44232794-a266-4e1f-afdb-d4d72e55b6ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-44232794-a266-4e1f-afdb-d4d72e55b6ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:04:51.598387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:51.599277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:51.600021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:51.600021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:51.600021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:51.600021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:51.600021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:04:51.600021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:04:51.600021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:04:51.600021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:04:51.600021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:04:51.600021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:04:51.600021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:04:51.600021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:04:51.602536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 39/74] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:04:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:04:51.604316 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44232794-a266-4e1f-afdb-d4d72e55b6ab None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:51.608220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94695003-dc34-481d-8bb0-b2732e4a6dc9 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:51.642984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:51.653118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7dad22-5824-4e7a-bc2d-04c11077002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:51.653856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dad22-5824-4e7a-bc2d-04c11077002a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.654045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dad22-5824-4e7a-bc2d-04c11077002a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.654321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dad22-5824-4e7a-bc2d-04c11077002a None admin] PATH_INFO: `/v3/users/f535a214d4d940d699e82a9e017b9c1f` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.669101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7dad22-5824-4e7a-bc2d-04c11077002a None admin] RBAC: Authorizing `identity:get_user(user_id=f535a214d4d940d699e82a9e017b9c1f)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:51.670559 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7dad22-5824-4e7a-bc2d-04c11077002a None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:51.677665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 36/75] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:04:51 2022] GET /identity/v3/users/f535a214d4d940d699e82a9e017b9c1f => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:51.677665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7dad22-5824-4e7a-bc2d-04c11077002a None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:51.735291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:51.748738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44232794-a266-4e1f-afdb-d4d72e55b6ab None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:51.791044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b6a43-f019-480d-87f6-22e379ff5001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:51.791549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b6a43-f019-480d-87f6-22e379ff5001 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.791825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b6a43-f019-480d-87f6-22e379ff5001 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.791917 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7dad22-5824-4e7a-bc2d-04c11077002a None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:51.792121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b6a43-f019-480d-87f6-22e379ff5001 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.802654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b6a43-f019-480d-87f6-22e379ff5001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-161b6a43-f019-480d-87f6-22e379ff5001 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:04:51.807606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:51.808630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 40/76] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:04:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:04:51.809590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b6a43-f019-480d-87f6-22e379ff5001 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:51.828521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfcec2-68f2-46d8-8873-9898f2d4b57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:51.829472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfcec2-68f2-46d8-8873-9898f2d4b57e None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.829472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfcec2-68f2-46d8-8873-9898f2d4b57e None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.829657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfcec2-68f2-46d8-8873-9898f2d4b57e None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.830285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bfcec2-68f2-46d8-8873-9898f2d4b57e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:51.830587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bfcec2-68f2-46d8-8873-9898f2d4b57e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:51.832561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bfcec2-68f2-46d8-8873-9898f2d4b57e None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:51.841026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 37/77] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:04:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:51.842113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfcec2-68f2-46d8-8873-9898f2d4b57e None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:51.898183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:51.919263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b6a43-f019-480d-87f6-22e379ff5001 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:51.944900 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfcec2-68f2-46d8-8873-9898f2d4b57e None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:51.954738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b7fba-b033-4870-936b-d7cdbf18c87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:51.955378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b7fba-b033-4870-936b-d7cdbf18c87f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.955662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b7fba-b033-4870-936b-d7cdbf18c87f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.955938 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b7fba-b033-4870-936b-d7cdbf18c87f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:51.957746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b7fba-b033-4870-936b-d7cdbf18c87f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:51.958429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:51.959225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b7fba-b033-4870-936b-d7cdbf18c87f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:51.960847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 41/78] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:04:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:51.965406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b7fba-b033-4870-936b-d7cdbf18c87f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:51.983983 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cda93a-83f5-49b4-bd1c-029150d5dd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:51.984610 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cda93a-83f5-49b4-bd1c-029150d5dd63 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:51.984903 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cda93a-83f5-49b4-bd1c-029150d5dd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:51.985284 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cda93a-83f5-49b4-bd1c-029150d5dd63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:52.055537 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cda93a-83f5-49b4-bd1c-029150d5dd63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:52.057033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cda93a-83f5-49b4-bd1c-029150d5dd63 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:52.059933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b7fba-b033-4870-936b-d7cdbf18c87f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:52.095130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79244aaf-c3fa-4e0d-947d-cf882cb0f36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:52.095761 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79244aaf-c3fa-4e0d-947d-cf882cb0f36b None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:52.096039 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79244aaf-c3fa-4e0d-947d-cf882cb0f36b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:52.096338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79244aaf-c3fa-4e0d-947d-cf882cb0f36b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:52.106409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79244aaf-c3fa-4e0d-947d-cf882cb0f36b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-79244aaf-c3fa-4e0d-947d-cf882cb0f36b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:04:52.111244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:52.112248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:52.113558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:52.113558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:52.113558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:52.113558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:52.113558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:04:52.113558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:04:52.113558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:04:52.113558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:04:52.113558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:04:52.113558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:04:52.113558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:04:52.113558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:04:52.113558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 42/79] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:04:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:04:52.115275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79244aaf-c3fa-4e0d-947d-cf882cb0f36b None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:52.124959 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:52.157905 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 38/80] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:04:52.158744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cda93a-83f5-49b4-bd1c-029150d5dd63 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:52.173110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:52.219352 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79244aaf-c3fa-4e0d-947d-cf882cb0f36b None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:52.252822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ebaeb-c577-4185-93b7-97574bffffaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:52.253457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ebaeb-c577-4185-93b7-97574bffffaf None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:52.253735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ebaeb-c577-4185-93b7-97574bffffaf None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:52.254012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ebaeb-c577-4185-93b7-97574bffffaf None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:52.254673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ebaeb-c577-4185-93b7-97574bffffaf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:52.254971 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ebaeb-c577-4185-93b7-97574bffffaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:52.256380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ebaeb-c577-4185-93b7-97574bffffaf None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:52.258703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cda93a-83f5-49b4-bd1c-029150d5dd63 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:52.264070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 43/81] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:04:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:52.264863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ebaeb-c577-4185-93b7-97574bffffaf None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:52.296803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab982-f717-44af-a87b-91351b4d1811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:52.297467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab982-f717-44af-a87b-91351b4d1811 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:52.297752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab982-f717-44af-a87b-91351b4d1811 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:52.298030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab982-f717-44af-a87b-91351b4d1811 None admin] PATH_INFO: `/v3/users/7afd1ca5c60c424cb58374006df79ec1` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:52.299150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cab982-f717-44af-a87b-91351b4d1811 None admin] RBAC: Authorizing `identity:get_user(user_id=7afd1ca5c60c424cb58374006df79ec1)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:52.300640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cab982-f717-44af-a87b-91351b4d1811 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:52.302417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 39/82] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:04:52 2022] GET /identity/v3/users/7afd1ca5c60c424cb58374006df79ec1 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:52.303042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab982-f717-44af-a87b-91351b4d1811 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:52.337027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:52.370596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ebaeb-c577-4185-93b7-97574bffffaf None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:52.382952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:52.402699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab982-f717-44af-a87b-91351b4d1811 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:52.405303 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d25265-7ba9-45a0-a982-e0bd61519f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:52.405919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d25265-7ba9-45a0-a982-e0bd61519f69 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:52.406204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d25265-7ba9-45a0-a982-e0bd61519f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:52.406483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d25265-7ba9-45a0-a982-e0bd61519f69 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:52.407135 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d25265-7ba9-45a0-a982-e0bd61519f69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:52.408714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d25265-7ba9-45a0-a982-e0bd61519f69 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:52.449890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77728385-455b-4a6f-a542-6138818a093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:52.450507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77728385-455b-4a6f-a542-6138818a093b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:52.450790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77728385-455b-4a6f-a542-6138818a093b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:52.451067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77728385-455b-4a6f-a542-6138818a093b None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:52.451738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77728385-455b-4a6f-a542-6138818a093b None admin] RBAC: Adding query filter params (name=member) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:52.452043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77728385-455b-4a6f-a542-6138818a093b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:52.453475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77728385-455b-4a6f-a542-6138818a093b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:52.462223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 40/83] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:04:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:52.463125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77728385-455b-4a6f-a542-6138818a093b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:52.489968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 44/84] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:04:52 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:04:52.490803 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d25265-7ba9-45a0-a982-e0bd61519f69 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:52.630513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d25265-7ba9-45a0-a982-e0bd61519f69 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:52.631372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77728385-455b-4a6f-a542-6138818a093b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:52.657166 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:52.673533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27df009-b207-40c9-a92b-3fdd04c66ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:52.674479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27df009-b207-40c9-a92b-3fdd04c66ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:52.674479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27df009-b207-40c9-a92b-3fdd04c66ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:52.674658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27df009-b207-40c9-a92b-3fdd04c66ce7 None admin] PATH_INFO: `/v3/users/f535a214d4d940d699e82a9e017b9c1f` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:52.677644 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27df009-b207-40c9-a92b-3fdd04c66ce7 None admin] RBAC: Authorizing `identity:get_user(user_id=f535a214d4d940d699e82a9e017b9c1f)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:52.678385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27df009-b207-40c9-a92b-3fdd04c66ce7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:52.680046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 45/85] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:04:52 2022] GET /identity/v3/users/f535a214d4d940d699e82a9e017b9c1f => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:52.681395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27df009-b207-40c9-a92b-3fdd04c66ce7 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:52.692295 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc853e-7a99-4959-8531-db57fc56f8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:52.692940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc853e-7a99-4959-8531-db57fc56f8ae None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:52.693223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc853e-7a99-4959-8531-db57fc56f8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:52.693505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc853e-7a99-4959-8531-db57fc56f8ae None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:52.695949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc853e-7a99-4959-8531-db57fc56f8ae None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:52.697473 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc853e-7a99-4959-8531-db57fc56f8ae None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:52.699156 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 41/86] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:04:52 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:52.699929 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc853e-7a99-4959-8531-db57fc56f8ae None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:52.714006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:52.815587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27df009-b207-40c9-a92b-3fdd04c66ce7 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:52.826354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc853e-7a99-4959-8531-db57fc56f8ae None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:52.866098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961d66f-12d6-401f-a8e6-c844a79f4383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:52.867388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961d66f-12d6-401f-a8e6-c844a79f4383 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:52.867923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961d66f-12d6-401f-a8e6-c844a79f4383 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:52.868506 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961d66f-12d6-401f-a8e6-c844a79f4383 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:52.875981 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7d42b-8e5f-444d-a42c-45caec372f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:52.876664 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7d42b-8e5f-444d-a42c-45caec372f65 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:52.876956 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7d42b-8e5f-444d-a42c-45caec372f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:52.877233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7d42b-8e5f-444d-a42c-45caec372f65 None admin] PATH_INFO: `/v3/users/f535a214d4d940d699e82a9e017b9c1f` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:52.878487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7d42b-8e5f-444d-a42c-45caec372f65 None admin] RBAC: Authorizing `identity:get_user(user_id=f535a214d4d940d699e82a9e017b9c1f)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:52.879960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7d42b-8e5f-444d-a42c-45caec372f65 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:52.881870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 42/87] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:04:52 2022] GET /identity/v3/users/f535a214d4d940d699e82a9e017b9c1f => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:52.882882 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7d42b-8e5f-444d-a42c-45caec372f65 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:52.957401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961d66f-12d6-401f-a8e6-c844a79f4383 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:52.959400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961d66f-12d6-401f-a8e6-c844a79f4383 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:52.962910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:52.979852 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:53.001577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7d42b-8e5f-444d-a42c-45caec372f65 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:53.045068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d9885-2b0b-4feb-9f56-be1caf60db31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:53.046055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d9885-2b0b-4feb-9f56-be1caf60db31 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:53.047090 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d9885-2b0b-4feb-9f56-be1caf60db31 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:53.047570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d9885-2b0b-4feb-9f56-be1caf60db31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:53.120018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 46/88] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:04:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:04:53.122311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961d66f-12d6-401f-a8e6-c844a79f4383 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:53.135605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d9885-2b0b-4feb-9f56-be1caf60db31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:53.137294 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d9885-2b0b-4feb-9f56-be1caf60db31 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:53.232689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:53.256457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 43/89] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:04:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:04:53.257387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d9885-2b0b-4feb-9f56-be1caf60db31 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:53.259081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:53.270244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961d66f-12d6-401f-a8e6-c844a79f4383 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:53.310888 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46da881-7843-4d41-ab62-0ecfbe7d77b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:53.311567 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46da881-7843-4d41-ab62-0ecfbe7d77b1 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:53.311929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46da881-7843-4d41-ab62-0ecfbe7d77b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:53.312282 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46da881-7843-4d41-ab62-0ecfbe7d77b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:53.313031 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46da881-7843-4d41-ab62-0ecfbe7d77b1 None admin] RBAC: Adding query filter params (role.id=4f10622fcf4c4f708cd6b652d886ba3e, scope.project.id=6d01d0d61d804d4393d51b2d191f652d, user.id=f535a214d4d940d699e82a9e017b9c1f) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:53.313347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46da881-7843-4d41-ab62-0ecfbe7d77b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:53.314931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46da881-7843-4d41-ab62-0ecfbe7d77b1 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:53.326327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 47/90] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:04:53 2022] GET /identity/v3/role_assignments?user.id=f535a214d4d940d699e82a9e017b9c1f&scope.project.id=6d01d0d61d804d4393d51b2d191f652d&role.id=4f10622fcf4c4f708cd6b652d886ba3e => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:53.326508 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46da881-7843-4d41-ab62-0ecfbe7d77b1 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:53.390577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d9885-2b0b-4feb-9f56-be1caf60db31 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:53.439795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43793c1d-d05a-4d9e-a3b5-64888e4c1bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:53.440964 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43793c1d-d05a-4d9e-a3b5-64888e4c1bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:53.441266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43793c1d-d05a-4d9e-a3b5-64888e4c1bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:53.441542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43793c1d-d05a-4d9e-a3b5-64888e4c1bc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:53.453857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46da881-7843-4d41-ab62-0ecfbe7d77b1 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:53.507074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370a7c3-3980-4083-bcfa-5abd70432cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:53.507649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370a7c3-3980-4083-bcfa-5abd70432cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:53.507936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370a7c3-3980-4083-bcfa-5abd70432cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:53.508242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370a7c3-3980-4083-bcfa-5abd70432cb3 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:53.511070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370a7c3-3980-4083-bcfa-5abd70432cb3 None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:53.515307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370a7c3-3980-4083-bcfa-5abd70432cb3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:53.518367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 48/91] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:04:53 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:53.523467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370a7c3-3980-4083-bcfa-5abd70432cb3 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:53.543335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:53.577544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:53.578324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43793c1d-d05a-4d9e-a3b5-64888e4c1bc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:53.579851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43793c1d-d05a-4d9e-a3b5-64888e4c1bc8 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:53.724196 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 44/92] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:04:53.725052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43793c1d-d05a-4d9e-a3b5-64888e4c1bc8 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:53.763823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370a7c3-3980-4083-bcfa-5abd70432cb3 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:53.840577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:53.845178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43793c1d-d05a-4d9e-a3b5-64888e4c1bc8 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:53.866514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd55d63-c290-4db2-bcec-ba8e33dd7399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:53.867528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd55d63-c290-4db2-bcec-ba8e33dd7399 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:53.869287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd55d63-c290-4db2-bcec-ba8e33dd7399 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:53.869772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd55d63-c290-4db2-bcec-ba8e33dd7399 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:53.871896 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd55d63-c290-4db2-bcec-ba8e33dd7399 None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:53.873787 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd55d63-c290-4db2-bcec-ba8e33dd7399 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:53.877398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 49/93] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:04:53 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:53.879682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd55d63-c290-4db2-bcec-ba8e33dd7399 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:53.921714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656aecf9-a47a-4f61-8334-6043fb0ab9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:53.923660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656aecf9-a47a-4f61-8334-6043fb0ab9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:53.927754 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656aecf9-a47a-4f61-8334-6043fb0ab9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:53.928271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656aecf9-a47a-4f61-8334-6043fb0ab9c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:53.929215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656aecf9-a47a-4f61-8334-6043fb0ab9c5 None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=6d01d0d61d804d4393d51b2d191f652d, user.id=7afd1ca5c60c424cb58374006df79ec1) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:53.929735 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656aecf9-a47a-4f61-8334-6043fb0ab9c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:53.932324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656aecf9-a47a-4f61-8334-6043fb0ab9c5 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:53.939207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 45/94] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:04:53 2022] GET /identity/v3/role_assignments?user.id=7afd1ca5c60c424cb58374006df79ec1&scope.project.id=6d01d0d61d804d4393d51b2d191f652d&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:53.941216 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656aecf9-a47a-4f61-8334-6043fb0ab9c5 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:54.101920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:54.115868 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656aecf9-a47a-4f61-8334-6043fb0ab9c5 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:54.121184 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd55d63-c290-4db2-bcec-ba8e33dd7399 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:54.176179 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95347fda-8707-4fe3-9b8c-c411e52c5f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:54.176799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95347fda-8707-4fe3-9b8c-c411e52c5f14 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:54.177086 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95347fda-8707-4fe3-9b8c-c411e52c5f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:54.177365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95347fda-8707-4fe3-9b8c-c411e52c5f14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:54.178100 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95347fda-8707-4fe3-9b8c-c411e52c5f14 None admin] RBAC: Adding query filter params (role.id=f1d8af7d158e4e02a6e77492178db259, scope.project.id=6d01d0d61d804d4393d51b2d191f652d, user.id=f535a214d4d940d699e82a9e017b9c1f) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:54.178417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95347fda-8707-4fe3-9b8c-c411e52c5f14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:54.179886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95347fda-8707-4fe3-9b8c-c411e52c5f14 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:54.188609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 46/95] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:04:53 2022] GET /identity/v3/role_assignments?user.id=f535a214d4d940d699e82a9e017b9c1f&scope.project.id=6d01d0d61d804d4393d51b2d191f652d&role.id=f1d8af7d158e4e02a6e77492178db259 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:54.194771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77280d9b-313f-4da7-bad2-5478d7893477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:54.195453 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77280d9b-313f-4da7-bad2-5478d7893477 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:54.195738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77280d9b-313f-4da7-bad2-5478d7893477 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:54.196020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77280d9b-313f-4da7-bad2-5478d7893477 None admin] PATH_INFO: `/v3/projects/45de5144b71346cca2ca4f287a2ba098` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:54.197347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77280d9b-313f-4da7-bad2-5478d7893477 None admin] RBAC: Authorizing `identity:get_project(project_id=45de5144b71346cca2ca4f287a2ba098)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:54.198860 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77280d9b-313f-4da7-bad2-5478d7893477 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:54.200675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 50/96] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:04:53 2022] GET /identity/v3/projects/45de5144b71346cca2ca4f287a2ba098 => generated 312 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:54.206915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95347fda-8707-4fe3-9b8c-c411e52c5f14 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:54.216571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:54.350648 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95347fda-8707-4fe3-9b8c-c411e52c5f14 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:54.407151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c734c62-8a01-4097-90b1-f67559845b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:54.408032 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c734c62-8a01-4097-90b1-f67559845b4d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:54.408354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c734c62-8a01-4097-90b1-f67559845b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:54.408635 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c734c62-8a01-4097-90b1-f67559845b4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:54.409456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c734c62-8a01-4097-90b1-f67559845b4d None admin] RBAC: Adding query filter params (role.id=4f10622fcf4c4f708cd6b652d886ba3e, scope.project.id=45de5144b71346cca2ca4f287a2ba098, user.id=f535a214d4d940d699e82a9e017b9c1f) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:54.409775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c734c62-8a01-4097-90b1-f67559845b4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:54.411256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c734c62-8a01-4097-90b1-f67559845b4d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:54.417728 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 47/97] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:04:54 2022] GET /identity/v3/role_assignments?user.id=f535a214d4d940d699e82a9e017b9c1f&scope.project.id=45de5144b71346cca2ca4f287a2ba098&role.id=4f10622fcf4c4f708cd6b652d886ba3e => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:54.914869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9641c-5b38-4299-a2d7-ff767f483fb6 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:54.915379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9641c-5b38-4299-a2d7-ff767f483fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:54.915452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9641c-5b38-4299-a2d7-ff767f483fb6 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:54.917111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 51/98] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:54.931194 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea611bbd-9aca-4235-917f-72c5ea31cf02 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:54.931867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea611bbd-9aca-4235-917f-72c5ea31cf02 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:54.931867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea611bbd-9aca-4235-917f-72c5ea31cf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:54.957266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ea611bbd-9aca-4235-917f-72c5ea31cf02 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:54.958582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ea611bbd-9aca-4235-917f-72c5ea31cf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:55.043314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:55.048132 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 48/99] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:55.201827 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7d500-6813-49a3-a731-2f9cd0824f3f None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:55.202134 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7d500-6813-49a3-a731-2f9cd0824f3f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:55.202488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7d500-6813-49a3-a731-2f9cd0824f3f None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:55.203971 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 52/100] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:55.210212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a77281-a2c7-42c7-bb1c-50b32858b641 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:55.210512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a77281-a2c7-42c7-bb1c-50b32858b641 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:55.210794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a77281-a2c7-42c7-bb1c-50b32858b641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:55.237799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-c9a77281-a2c7-42c7-bb1c-50b32858b641 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:55.239124 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c9a77281-a2c7-42c7-bb1c-50b32858b641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:55.309463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 49/101] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:55.471136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7d500-6813-49a3-a731-2f9cd0824f3f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:55.605917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7d500-6813-49a3-a731-2f9cd0824f3f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:55.652715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a77281-a2c7-42c7-bb1c-50b32858b641 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:55.656834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea90ea8-919e-4451-a3b0-2dd2d6bd4280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:55.657547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea90ea8-919e-4451-a3b0-2dd2d6bd4280 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:55.658721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea90ea8-919e-4451-a3b0-2dd2d6bd4280 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:55.659007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea90ea8-919e-4451-a3b0-2dd2d6bd4280 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:55.664864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea90ea8-919e-4451-a3b0-2dd2d6bd4280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:55.666348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea90ea8-919e-4451-a3b0-2dd2d6bd4280 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:55.668551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 53/102] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:04:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:55.675169 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea90ea8-919e-4451-a3b0-2dd2d6bd4280 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:55.744606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:55.895127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea90ea8-919e-4451-a3b0-2dd2d6bd4280 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:55.953616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b1f3b-29b0-4b2f-9823-9040d04589a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:55.954527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b1f3b-29b0-4b2f-9823-9040d04589a7 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:55.954821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b1f3b-29b0-4b2f-9823-9040d04589a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:55.955100 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b1f3b-29b0-4b2f-9823-9040d04589a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:55.955770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b1f3b-29b0-4b2f-9823-9040d04589a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:55.958319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b1f3b-29b0-4b2f-9823-9040d04589a7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:55.962109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a77281-a2c7-42c7-bb1c-50b32858b641 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:56.038401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:56.044612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0b4da-a487-4371-9f9f-8e507da26f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:56.045252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0b4da-a487-4371-9f9f-8e507da26f86 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:56.045538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0b4da-a487-4371-9f9f-8e507da26f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:56.045817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0b4da-a487-4371-9f9f-8e507da26f86 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:56.056750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0b4da-a487-4371-9f9f-8e507da26f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-7bb0b4da-a487-4371-9f9f-8e507da26f86 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:04:56.061725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:56.063313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:56.064480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:56.064480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:56.064480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:56.064480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:56.064480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:04:56.064480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:04:56.064480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:04:56.064480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:04:56.064480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:04:56.064480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:04:56.064480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:04:56.064480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:04:56.064480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 50/103] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:04:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:04:56.065536 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3784958-377d-4907-b01e-95522d743798 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:56.065833 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3784958-377d-4907-b01e-95522d743798 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:56.066111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3784958-377d-4907-b01e-95522d743798 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:56.067410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 51/104] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:56.072370 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3784958-377d-4907-b01e-95522d743798 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:56.098696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 54/105] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:04:55 2022] POST /identity/v3/projects => generated 302 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:04:56.100070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e33db-83a9-43c1-89dd-9f91536cf3f8 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:56.100401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e33db-83a9-43c1-89dd-9f91536cf3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:56.100739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e33db-83a9-43c1-89dd-9f91536cf3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:56.140692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-1b2e33db-83a9-43c1-89dd-9f91536cf3f8 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:56.142270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1b2e33db-83a9-43c1-89dd-9f91536cf3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:56.258763 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3784958-377d-4907-b01e-95522d743798 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:56.294657 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 55/106] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:56.328750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42baccf9-0872-4ba9-b1e6-590d8e7c9ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:56.329402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42baccf9-0872-4ba9-b1e6-590d8e7c9ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:56.329747 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42baccf9-0872-4ba9-b1e6-590d8e7c9ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:56.330029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42baccf9-0872-4ba9-b1e6-590d8e7c9ca3 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:56.330725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42baccf9-0872-4ba9-b1e6-590d8e7c9ca3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:56.331092 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42baccf9-0872-4ba9-b1e6-590d8e7c9ca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:56.332681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42baccf9-0872-4ba9-b1e6-590d8e7c9ca3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:56.348909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 52/107] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:04:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:04:56.354880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e33db-83a9-43c1-89dd-9f91536cf3f8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:56.372297 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:56.498902 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e33db-83a9-43c1-89dd-9f91536cf3f8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:56.545998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9debe68-4e9a-439f-ba13-2ee8abf51207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:56.546565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9debe68-4e9a-439f-ba13-2ee8abf51207 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:56.546849 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9debe68-4e9a-439f-ba13-2ee8abf51207 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:56.547127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9debe68-4e9a-439f-ba13-2ee8abf51207 None admin] PATH_INFO: `/v3/users/8d6ab68412ec4430998b625c98741ff8` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:56.566805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9debe68-4e9a-439f-ba13-2ee8abf51207 None admin] RBAC: Authorizing `identity:get_user(user_id=8d6ab68412ec4430998b625c98741ff8)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:56.568373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9debe68-4e9a-439f-ba13-2ee8abf51207 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:56.570393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 56/108] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:04:56 2022] GET /identity/v3/users/8d6ab68412ec4430998b625c98741ff8 => generated 289 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:56.576102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42baccf9-0872-4ba9-b1e6-590d8e7c9ca3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:56.663415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9debe68-4e9a-439f-ba13-2ee8abf51207 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:56.684575 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:56.776332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42baccf9-0872-4ba9-b1e6-590d8e7c9ca3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:56.809017 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:56.822073 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31014ca5-aead-4c6b-99f9-8c3de520788b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:56.822759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31014ca5-aead-4c6b-99f9-8c3de520788b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:56.823049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31014ca5-aead-4c6b-99f9-8c3de520788b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:56.823497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31014ca5-aead-4c6b-99f9-8c3de520788b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:56.874798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9debe68-4e9a-439f-ba13-2ee8abf51207 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:56.939109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75bc47-69b8-461a-8329-b42063c5cce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:56.939763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75bc47-69b8-461a-8329-b42063c5cce0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:56.940045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75bc47-69b8-461a-8329-b42063c5cce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:56.940406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75bc47-69b8-461a-8329-b42063c5cce0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:56.946791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31014ca5-aead-4c6b-99f9-8c3de520788b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:56.948354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31014ca5-aead-4c6b-99f9-8c3de520788b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:56.963505 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75bc47-69b8-461a-8329-b42063c5cce0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-eb75bc47-69b8-461a-8329-b42063c5cce0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:04:56.970338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:56.971274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:56.972230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:56.972230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:56.972230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:56.972230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:56.972230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:04:56.972230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:04:56.972230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:04:56.972230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:04:56.972230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:04:56.972230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:04:56.972230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:04:56.972230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:04:56.972230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 57/109] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:04:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:04:56.972846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e050f4f-e74f-478c-8c59-6d1ec4902068 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:56.972846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e050f4f-e74f-478c-8c59-6d1ec4902068 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:56.973718 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e050f4f-e74f-478c-8c59-6d1ec4902068 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:56.975728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 58/110] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:56.976553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e050f4f-e74f-478c-8c59-6d1ec4902068 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:57.025374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:57.081487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 53/111] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:04:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:04:57.083456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1db39-f1db-4b44-80de-8443e00df996 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:57.084053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1db39-f1db-4b44-80de-8443e00df996 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:57.084694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1db39-f1db-4b44-80de-8443e00df996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:57.113542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-91c1db39-f1db-4b44-80de-8443e00df996 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:57.114869 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-91c1db39-f1db-4b44-80de-8443e00df996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:57.144990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e050f4f-e74f-478c-8c59-6d1ec4902068 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:57.184895 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2dddc1-ca8e-44a3-a120-d776abe98dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:57.185893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2dddc1-ca8e-44a3-a120-d776abe98dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:57.186370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2dddc1-ca8e-44a3-a120-d776abe98dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:57.186834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2dddc1-ca8e-44a3-a120-d776abe98dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:57.187693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2dddc1-ca8e-44a3-a120-d776abe98dc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:57.188238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2dddc1-ca8e-44a3-a120-d776abe98dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:57.189912 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2dddc1-ca8e-44a3-a120-d776abe98dc2 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:57.192492 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:57.201135 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 59/112] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:04:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:57.202490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2dddc1-ca8e-44a3-a120-d776abe98dc2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:57.279914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 54/113] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:57.280635 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1db39-f1db-4b44-80de-8443e00df996 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:57.316018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2dddc1-ca8e-44a3-a120-d776abe98dc2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:57.357619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54235f8-0c24-4bcf-8660-b25c7c347642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:57.357794 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54235f8-0c24-4bcf-8660-b25c7c347642 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:57.358019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54235f8-0c24-4bcf-8660-b25c7c347642 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:57.358275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54235f8-0c24-4bcf-8660-b25c7c347642 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:57.359494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54235f8-0c24-4bcf-8660-b25c7c347642 None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:57.361382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54235f8-0c24-4bcf-8660-b25c7c347642 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:57.363682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 60/114] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:04:57 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:57.369825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54235f8-0c24-4bcf-8660-b25c7c347642 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:57.471448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:57.481397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1db39-f1db-4b44-80de-8443e00df996 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:57.483202 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:57.521367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff421d6-71ed-4f7f-9ca9-67da55f3069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:57.522240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff421d6-71ed-4f7f-9ca9-67da55f3069b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:57.522671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff421d6-71ed-4f7f-9ca9-67da55f3069b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:57.523137 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff421d6-71ed-4f7f-9ca9-67da55f3069b None admin] PATH_INFO: `/v3/users/f535a214d4d940d699e82a9e017b9c1f` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:57.524480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff421d6-71ed-4f7f-9ca9-67da55f3069b None admin] RBAC: Authorizing `identity:get_user(user_id=f535a214d4d940d699e82a9e017b9c1f)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:57.526407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff421d6-71ed-4f7f-9ca9-67da55f3069b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:57.528234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 55/115] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:04:57 2022] GET /identity/v3/users/f535a214d4d940d699e82a9e017b9c1f => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:57.529603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21390251-f36d-4052-8720-f99296d78889 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:57.530054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21390251-f36d-4052-8720-f99296d78889 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:57.530475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21390251-f36d-4052-8720-f99296d78889 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:57.532118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 56/116] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:57.536435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54235f8-0c24-4bcf-8660-b25c7c347642 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:57.538544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7796372-e724-4ebf-bf72-9ddd9d3bca72 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:57.539080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7796372-e724-4ebf-bf72-9ddd9d3bca72 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:57.539558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7796372-e724-4ebf-bf72-9ddd9d3bca72 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:57.541560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 57/117] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:57.542497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7796372-e724-4ebf-bf72-9ddd9d3bca72 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:57.579974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d397664-7f7c-4905-a33b-88919da2fd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:57.581968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d397664-7f7c-4905-a33b-88919da2fd44 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:57.582464 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d397664-7f7c-4905-a33b-88919da2fd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:57.582959 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d397664-7f7c-4905-a33b-88919da2fd44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:57.583858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d397664-7f7c-4905-a33b-88919da2fd44 None admin] RBAC: Adding query filter params (role.id=7da1608958c94a6dbb98eaf955981d2d, scope.project.id=6d01d0d61d804d4393d51b2d191f652d, user.id=8d6ab68412ec4430998b625c98741ff8) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:57.584409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d397664-7f7c-4905-a33b-88919da2fd44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:57.587911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d397664-7f7c-4905-a33b-88919da2fd44 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:57.592837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 61/118] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:04:57 2022] GET /identity/v3/role_assignments?user.id=8d6ab68412ec4430998b625c98741ff8&scope.project.id=6d01d0d61d804d4393d51b2d191f652d&role.id=7da1608958c94a6dbb98eaf955981d2d => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:04:57.594374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78145f49-efa3-4375-b915-56ba39f517ab None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:57.594970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78145f49-efa3-4375-b915-56ba39f517ab None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:57.595452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78145f49-efa3-4375-b915-56ba39f517ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:57.622617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-78145f49-efa3-4375-b915-56ba39f517ab None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:57.622617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-78145f49-efa3-4375-b915-56ba39f517ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:57.702198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7796372-e724-4ebf-bf72-9ddd9d3bca72 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:57.730296 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 62/119] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:57.730296 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba398dca-8713-4a5f-a085-e7601f80ffa8 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:57.730296 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba398dca-8713-4a5f-a085-e7601f80ffa8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:57.730296 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba398dca-8713-4a5f-a085-e7601f80ffa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:57.750898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-ba398dca-8713-4a5f-a085-e7601f80ffa8 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:57.752674 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ba398dca-8713-4a5f-a085-e7601f80ffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:57.776103 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e1172-1817-427b-a568-26dec4f0f5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:57.777159 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e1172-1817-427b-a568-26dec4f0f5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:57.777445 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e1172-1817-427b-a568-26dec4f0f5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:57.777723 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e1172-1817-427b-a568-26dec4f0f5d8 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:57.781543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e1172-1817-427b-a568-26dec4f0f5d8 None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:57.783404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e1172-1817-427b-a568-26dec4f0f5d8 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:57.786062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 58/120] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:04:57 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:57.786681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e1172-1817-427b-a568-26dec4f0f5d8 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:57.816119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:57.832585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:57.844757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 63/121] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:57.846410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba398dca-8713-4a5f-a085-e7601f80ffa8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:57.963605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba398dca-8713-4a5f-a085-e7601f80ffa8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:57.995023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e1172-1817-427b-a568-26dec4f0f5d8 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:58.069824 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e1f41-99bb-498a-9589-56bc5ce8465e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:58.070447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e1f41-99bb-498a-9589-56bc5ce8465e None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:58.070734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e1f41-99bb-498a-9589-56bc5ce8465e None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:58.071014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e1f41-99bb-498a-9589-56bc5ce8465e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:58.076441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93edc62-5bae-48c3-8433-a3a961e98f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:58.077164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93edc62-5bae-48c3-8433-a3a961e98f7c None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:58.077454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93edc62-5bae-48c3-8433-a3a961e98f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:58.077734 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93edc62-5bae-48c3-8433-a3a961e98f7c None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d/users/f535a214d4d940d699e82a9e017b9c1f/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:58.082040 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93edc62-5bae-48c3-8433-a3a961e98f7c None admin] RBAC: Authorizing `identity:create_grant(project_id=6d01d0d61d804d4393d51b2d191f652d, user_id=f535a214d4d940d699e82a9e017b9c1f, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:58.083497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93edc62-5bae-48c3-8433-a3a961e98f7c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:58.093790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e1f41-99bb-498a-9589-56bc5ce8465e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-776e1f41-99bb-498a-9589-56bc5ce8465e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:04:58.099385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:58.100279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:58.101087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:58.101087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:58.101087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:58.101087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:58.101087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:04:58.101087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:04:58.101087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:04:58.101087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:04:58.101087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:04:58.101087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:04:58.101087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:04:58.101087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:04:58.101087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 59/122] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:04:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:04:58.101696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e1f41-99bb-498a-9589-56bc5ce8465e None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:58.171385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 64/123] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:04:57 2022] PUT /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d/users/f535a214d4d940d699e82a9e017b9c1f/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:04:58.176596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93edc62-5bae-48c3-8433-a3a961e98f7c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:58.197248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:58.220834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e1f41-99bb-498a-9589-56bc5ce8465e None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:58.277937 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:58.298231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d42488-4a98-4067-8d6c-4f0c86328f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:58.299252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d42488-4a98-4067-8d6c-4f0c86328f0a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:58.300346 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d42488-4a98-4067-8d6c-4f0c86328f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:58.300904 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d42488-4a98-4067-8d6c-4f0c86328f0a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:58.323553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d42488-4a98-4067-8d6c-4f0c86328f0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-93d42488-4a98-4067-8d6c-4f0c86328f0a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:04:58.329172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:58.330123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:58.331417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:58.331417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:58.331417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:58.331417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:58.331417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:04:58.331417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:04:58.331417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:04:58.331417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:04:58.331417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:04:58.331417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:04:58.331417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:04:58.331417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:04:58.342522 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93edc62-5bae-48c3-8433-a3a961e98f7c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:58.347738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 60/124] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:04:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:04:58.347738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d42488-4a98-4067-8d6c-4f0c86328f0a None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:58.387738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71490c-2b80-4400-b5bb-fd8e89a8a733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:58.388647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71490c-2b80-4400-b5bb-fd8e89a8a733 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:58.389071 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71490c-2b80-4400-b5bb-fd8e89a8a733 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:58.389518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71490c-2b80-4400-b5bb-fd8e89a8a733 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:58.390333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71490c-2b80-4400-b5bb-fd8e89a8a733 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:58.390785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71490c-2b80-4400-b5bb-fd8e89a8a733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:58.392423 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71490c-2b80-4400-b5bb-fd8e89a8a733 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:58.402395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 65/125] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:04:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:58.403497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71490c-2b80-4400-b5bb-fd8e89a8a733 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:58.459798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d42488-4a98-4067-8d6c-4f0c86328f0a None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:58.497839 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad3236-3701-4c05-ac7a-862f37515ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:58.498716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad3236-3701-4c05-ac7a-862f37515ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:58.498823 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad3236-3701-4c05-ac7a-862f37515ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:58.499037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad3236-3701-4c05-ac7a-862f37515ae8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:58.507660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:58.512207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad3236-3701-4c05-ac7a-862f37515ae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:04:58.512328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71490c-2b80-4400-b5bb-fd8e89a8a733 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:58.514892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-44ad3236-3701-4c05-ac7a-862f37515ae8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:04:58.518190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:04:58.519315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:04:58.520074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:04:58.520074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:04:58.520074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:04:58.520074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:04:58.520074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:04:58.520074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:04:58.520074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:04:58.520074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:04:58.520074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:04:58.520074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:04:58.520074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:04:58.520074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:04:58.522446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 61/126] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:04:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:04:58.523432 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad3236-3701-4c05-ac7a-862f37515ae8 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:58.558138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ee5a3-740c-4edf-aaa4-8da8a30a6c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:58.558719 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ee5a3-740c-4edf-aaa4-8da8a30a6c8f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:58.559099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ee5a3-740c-4edf-aaa4-8da8a30a6c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:58.559476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ee5a3-740c-4edf-aaa4-8da8a30a6c8f None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:58.560217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ee5a3-740c-4edf-aaa4-8da8a30a6c8f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:58.560600 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ee5a3-740c-4edf-aaa4-8da8a30a6c8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:58.562084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ee5a3-740c-4edf-aaa4-8da8a30a6c8f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:58.582333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 66/127] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:04:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:58.583346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ee5a3-740c-4edf-aaa4-8da8a30a6c8f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:58.637369 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad3236-3701-4c05-ac7a-862f37515ae8 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:58.689508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030197f2-6a3e-40b1-b95b-38131df59291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:58.691049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030197f2-6a3e-40b1-b95b-38131df59291 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:58.691529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030197f2-6a3e-40b1-b95b-38131df59291 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:58.692021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030197f2-6a3e-40b1-b95b-38131df59291 None admin] PATH_INFO: `/v3/users/7afd1ca5c60c424cb58374006df79ec1` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:58.696891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030197f2-6a3e-40b1-b95b-38131df59291 None admin] RBAC: Authorizing `identity:get_user(user_id=7afd1ca5c60c424cb58374006df79ec1)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:58.698563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030197f2-6a3e-40b1-b95b-38131df59291 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:58.700362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 62/128] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:04:58 2022] GET /identity/v3/users/7afd1ca5c60c424cb58374006df79ec1 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:58.701305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030197f2-6a3e-40b1-b95b-38131df59291 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:58.748689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ee5a3-740c-4edf-aaa4-8da8a30a6c8f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:58.792047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34123a-a710-4963-9bd2-c474d850a4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:58.792857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34123a-a710-4963-9bd2-c474d850a4fd None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:58.793217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34123a-a710-4963-9bd2-c474d850a4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:58.793929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34123a-a710-4963-9bd2-c474d850a4fd None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:58.794683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34123a-a710-4963-9bd2-c474d850a4fd None admin] RBAC: Adding query filter params (name=member) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:04:58.795063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34123a-a710-4963-9bd2-c474d850a4fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:58.796908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34123a-a710-4963-9bd2-c474d850a4fd None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:58.807207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 67/129] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:04:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:58.813051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34123a-a710-4963-9bd2-c474d850a4fd None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:58.839769 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:58.850995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:58.907254 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030197f2-6a3e-40b1-b95b-38131df59291 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:58.929586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34123a-a710-4963-9bd2-c474d850a4fd None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:58.943946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86984f61-80ee-4d23-90d2-e8973f1ee0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:58.944623 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86984f61-80ee-4d23-90d2-e8973f1ee0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:58.944908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86984f61-80ee-4d23-90d2-e8973f1ee0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:58.945187 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86984f61-80ee-4d23-90d2-e8973f1ee0f0 None admin] PATH_INFO: `/v3/users/f535a214d4d940d699e82a9e017b9c1f` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:58.946752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86984f61-80ee-4d23-90d2-e8973f1ee0f0 None admin] RBAC: Authorizing `identity:get_user(user_id=f535a214d4d940d699e82a9e017b9c1f)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:58.948009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86984f61-80ee-4d23-90d2-e8973f1ee0f0 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:58.949767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 63/130] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:04:58 2022] GET /identity/v3/users/f535a214d4d940d699e82a9e017b9c1f => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:58.955075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86984f61-80ee-4d23-90d2-e8973f1ee0f0 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:58.974701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c3cc0-64e3-40bd-9da1-e042dbebbbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:58.976029 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c3cc0-64e3-40bd-9da1-e042dbebbbd7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:58.976675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c3cc0-64e3-40bd-9da1-e042dbebbbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:58.977204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c3cc0-64e3-40bd-9da1-e042dbebbbd7 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:58.978674 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c3cc0-64e3-40bd-9da1-e042dbebbbd7 None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:58.980468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c3cc0-64e3-40bd-9da1-e042dbebbbd7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:58.982430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 68/131] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:04:58 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:58.983935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823cf6d-74ba-4f0f-b16e-cfe30e8d237f None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:58.984501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823cf6d-74ba-4f0f-b16e-cfe30e8d237f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:58.985173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823cf6d-74ba-4f0f-b16e-cfe30e8d237f None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:58.986942 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 69/132] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:58.988581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823cf6d-74ba-4f0f-b16e-cfe30e8d237f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:59.073104 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86984f61-80ee-4d23-90d2-e8973f1ee0f0 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:59.109583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612480b0-1cc9-4abd-adfc-871bd97298b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:59.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612480b0-1cc9-4abd-adfc-871bd97298b8 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:59.110788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612480b0-1cc9-4abd-adfc-871bd97298b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:59.111066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612480b0-1cc9-4abd-adfc-871bd97298b8 None admin] PATH_INFO: `/v3/users/f535a214d4d940d699e82a9e017b9c1f` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:59.112843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612480b0-1cc9-4abd-adfc-871bd97298b8 None admin] RBAC: Authorizing `identity:get_user(user_id=f535a214d4d940d699e82a9e017b9c1f)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:59.114357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612480b0-1cc9-4abd-adfc-871bd97298b8 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:59.115943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 64/133] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:04:58 2022] GET /identity/v3/users/f535a214d4d940d699e82a9e017b9c1f => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:59.117150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94bf76-567d-4d77-83a6-fc4e0be4fee3 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:59.117439 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94bf76-567d-4d77-83a6-fc4e0be4fee3 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:59.117717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94bf76-567d-4d77-83a6-fc4e0be4fee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:59.124063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:59.132275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:59.139086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823cf6d-74ba-4f0f-b16e-cfe30e8d237f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:59.143483 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ab94bf76-567d-4d77-83a6-fc4e0be4fee3 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:59.144985 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ab94bf76-567d-4d77-83a6-fc4e0be4fee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:04:59.175886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00eb3d-e417-49f3-a678-2bc9a5801b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:59.176640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00eb3d-e417-49f3-a678-2bc9a5801b31 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:59.177044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00eb3d-e417-49f3-a678-2bc9a5801b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:59.177387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00eb3d-e417-49f3-a678-2bc9a5801b31 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:59.178642 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e00eb3d-e417-49f3-a678-2bc9a5801b31 None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:59.180409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e00eb3d-e417-49f3-a678-2bc9a5801b31 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:59.183520 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 70/134] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:04:58 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:59.183520 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00eb3d-e417-49f3-a678-2bc9a5801b31 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:59.233171 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 65/135] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:04:59.240546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94bf76-567d-4d77-83a6-fc4e0be4fee3 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:59.354454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00eb3d-e417-49f3-a678-2bc9a5801b31 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:59.397774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94bf76-567d-4d77-83a6-fc4e0be4fee3 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:59.422834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6ff07-36ba-45b7-8395-7f08e32acb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:59.423513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6ff07-36ba-45b7-8395-7f08e32acb7c None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:59.423798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6ff07-36ba-45b7-8395-7f08e32acb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:59.424081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6ff07-36ba-45b7-8395-7f08e32acb7c None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d/users/7afd1ca5c60c424cb58374006df79ec1/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:59.430029 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6ff07-36ba-45b7-8395-7f08e32acb7c None admin] RBAC: Authorizing `identity:create_grant(project_id=6d01d0d61d804d4393d51b2d191f652d, user_id=7afd1ca5c60c424cb58374006df79ec1, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:59.431489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6ff07-36ba-45b7-8395-7f08e32acb7c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:59.443716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:59.448393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2def95-110c-48c0-ad45-e206efb5fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:59.449169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2def95-110c-48c0-ad45-e206efb5fa1d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:59.449462 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2def95-110c-48c0-ad45-e206efb5fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:59.449750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2def95-110c-48c0-ad45-e206efb5fa1d None admin] PATH_INFO: `/v3/projects/45de5144b71346cca2ca4f287a2ba098` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:59.451112 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2def95-110c-48c0-ad45-e206efb5fa1d None admin] RBAC: Authorizing `identity:get_project(project_id=45de5144b71346cca2ca4f287a2ba098)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:59.452699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2def95-110c-48c0-ad45-e206efb5fa1d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:59.454955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 66/136] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:04:59 2022] GET /identity/v3/projects/45de5144b71346cca2ca4f287a2ba098 => generated 312 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:04:59.458109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2def95-110c-48c0-ad45-e206efb5fa1d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:59.570395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2def95-110c-48c0-ad45-e206efb5fa1d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:59.609354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 71/137] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:04:59 2022] PUT /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d/users/7afd1ca5c60c424cb58374006df79ec1/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:04:59.610290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6ff07-36ba-45b7-8395-7f08e32acb7c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:59.629801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ed296-d0cd-4ac7-a8e5-a5060c9d6da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:59.632605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ed296-d0cd-4ac7-a8e5-a5060c9d6da9 None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:59.632903 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ed296-d0cd-4ac7-a8e5-a5060c9d6da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:59.633166 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ed296-d0cd-4ac7-a8e5-a5060c9d6da9 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d/users/f535a214d4d940d699e82a9e017b9c1f/roles/f1d8af7d158e4e02a6e77492178db259` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:59.640328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ed296-d0cd-4ac7-a8e5-a5060c9d6da9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d01d0d61d804d4393d51b2d191f652d, user_id=f535a214d4d940d699e82a9e017b9c1f, role_id=f1d8af7d158e4e02a6e77492178db259)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:59.644869 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ed296-d0cd-4ac7-a8e5-a5060c9d6da9 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:59.668184 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:59.759242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6ff07-36ba-45b7-8395-7f08e32acb7c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:04:59.795518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223cc54-dbdb-4593-b95d-4cee227577bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:04:59.796220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223cc54-dbdb-4593-b95d-4cee227577bb None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:59.796511 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223cc54-dbdb-4593-b95d-4cee227577bb None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:59.796791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223cc54-dbdb-4593-b95d-4cee227577bb None admin] PATH_INFO: `/v3/projects/45de5144b71346cca2ca4f287a2ba098/users/f535a214d4d940d699e82a9e017b9c1f/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:59.798943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223cc54-dbdb-4593-b95d-4cee227577bb None admin] RBAC: Authorizing `identity:create_grant(project_id=45de5144b71346cca2ca4f287a2ba098, user_id=f535a214d4d940d699e82a9e017b9c1f, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:04:59.800953 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223cc54-dbdb-4593-b95d-4cee227577bb None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:04:59.820713 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 67/138] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:04:59 2022] PUT /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d/users/f535a214d4d940d699e82a9e017b9c1f/roles/f1d8af7d158e4e02a6e77492178db259 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:04:59.821643 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ed296-d0cd-4ac7-a8e5-a5060c9d6da9 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:04:59.879177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 72/139] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:04:59 2022] PUT /identity/v3/projects/45de5144b71346cca2ca4f287a2ba098/users/f535a214d4d940d699e82a9e017b9c1f/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:04:59.893214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd289d6-f3dd-4f73-91b1-4a4be3bfa732 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:59.894087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd289d6-f3dd-4f73-91b1-4a4be3bfa732 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:59.894375 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd289d6-f3dd-4f73-91b1-4a4be3bfa732 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:59.896145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 73/140] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:04:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:04:59.902646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e6431-bc79-478f-85a0-4592fb5dbef3 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:04:59.902965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e6431-bc79-478f-85a0-4592fb5dbef3 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:04:59.903230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e6431-bc79-478f-85a0-4592fb5dbef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:04:59.929593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:04:59.939670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-c32e6431-bc79-478f-85a0-4592fb5dbef3 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:04:59.945531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c32e6431-bc79-478f-85a0-4592fb5dbef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:00.023196 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ed296-d0cd-4ac7-a8e5-a5060c9d6da9 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:00.062679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 74/141] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:00.093440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72065db9-7bbe-43ec-bfd3-77df6e3951ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:00.094234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72065db9-7bbe-43ec-bfd3-77df6e3951ee None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:00.094525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72065db9-7bbe-43ec-bfd3-77df6e3951ee None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:00.094806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72065db9-7bbe-43ec-bfd3-77df6e3951ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:00.097613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72065db9-7bbe-43ec-bfd3-77df6e3951ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:00.099121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72065db9-7bbe-43ec-bfd3-77df6e3951ee None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:00.100830 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 68/142] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:04:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:00.106369 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e6431-bc79-478f-85a0-4592fb5dbef3 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:00.133702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:00.234767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e6431-bc79-478f-85a0-4592fb5dbef3 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:00.272042 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efced96-5f7d-4587-8375-666b749d7808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:00.272825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efced96-5f7d-4587-8375-666b749d7808 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:00.273204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efced96-5f7d-4587-8375-666b749d7808 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:00.273580 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efced96-5f7d-4587-8375-666b749d7808 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:00.274351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efced96-5f7d-4587-8375-666b749d7808 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:00.276193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efced96-5f7d-4587-8375-666b749d7808 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:00.372942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72065db9-7bbe-43ec-bfd3-77df6e3951ee None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:00.435820 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 75/143] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:05:00 2022] POST /identity/v3/users => generated 283 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:05:00.513095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72065db9-7bbe-43ec-bfd3-77df6e3951ee None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:00.555980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:00.564365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27bc532-b30e-4681-ada7-15e7083d60c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:00.565031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bc532-b30e-4681-ada7-15e7083d60c6 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:00.565407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bc532-b30e-4681-ada7-15e7083d60c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:00.565693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bc532-b30e-4681-ada7-15e7083d60c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:00.577565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27bc532-b30e-4681-ada7-15e7083d60c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-e27bc532-b30e-4681-ada7-15e7083d60c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:00.583232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:00.584338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:00.585243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:00.585243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:00.585243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:00.585243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:00.585243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:00.585243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:00.585243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:00.585243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:00.585243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:00.585243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:00.585243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:00.585243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:00.585243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 69/144] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:00.590368 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efced96-5f7d-4587-8375-666b749d7808 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:00.688590 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0962728-1700-449e-8fbb-46a7a1bbf4f6 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:00.688805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0962728-1700-449e-8fbb-46a7a1bbf4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:00.689092 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0962728-1700-449e-8fbb-46a7a1bbf4f6 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:00.690587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 70/145] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:00.696065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f39cbe-941f-41ea-972a-2f739a8e73b3 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:00.696383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f39cbe-941f-41ea-972a-2f739a8e73b3 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:00.696677 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f39cbe-941f-41ea-972a-2f739a8e73b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:00.722956 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:00.725172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-c7f39cbe-941f-41ea-972a-2f739a8e73b3 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:00.731314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c7f39cbe-941f-41ea-972a-2f739a8e73b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:00.787225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efced96-5f7d-4587-8375-666b749d7808 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:00.839705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fbd42-a2c1-4783-bfef-45a9965a2487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:00.840355 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fbd42-a2c1-4783-bfef-45a9965a2487 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:00.840636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fbd42-a2c1-4783-bfef-45a9965a2487 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:00.840908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fbd42-a2c1-4783-bfef-45a9965a2487 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:00.841580 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fbd42-a2c1-4783-bfef-45a9965a2487 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:00.841882 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fbd42-a2c1-4783-bfef-45a9965a2487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:00.843286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fbd42-a2c1-4783-bfef-45a9965a2487 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:00.863556 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 76/146] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:00.870632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fbd42-a2c1-4783-bfef-45a9965a2487 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:00.871600 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 71/147] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:05:00.977652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fbd42-a2c1-4783-bfef-45a9965a2487 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:01.009141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:01.015034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c8061-0343-453a-862d-dd55e157db8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:01.015682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c8061-0343-453a-862d-dd55e157db8e None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:01.015963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c8061-0343-453a-862d-dd55e157db8e None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:01.016266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c8061-0343-453a-862d-dd55e157db8e None admin] PATH_INFO: `/v3/users/8d6ab68412ec4430998b625c98741ff8` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:01.017518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c8061-0343-453a-862d-dd55e157db8e None admin] RBAC: Authorizing `identity:get_user(user_id=8d6ab68412ec4430998b625c98741ff8)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:01.019452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c8061-0343-453a-862d-dd55e157db8e None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:01.021532 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 77/148] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:00 2022] GET /identity/v3/users/8d6ab68412ec4430998b625c98741ff8 => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:01.027433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f39cbe-941f-41ea-972a-2f739a8e73b3 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:01.141165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f39cbe-941f-41ea-972a-2f739a8e73b3 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:01.161540 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:01.180970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ec6e8-1fcd-4afa-a60a-2458d65a665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:01.181644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ec6e8-1fcd-4afa-a60a-2458d65a665c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:01.181936 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ec6e8-1fcd-4afa-a60a-2458d65a665c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:01.182212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ec6e8-1fcd-4afa-a60a-2458d65a665c None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:01.183365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ec6e8-1fcd-4afa-a60a-2458d65a665c None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:01.184841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ec6e8-1fcd-4afa-a60a-2458d65a665c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:01.186472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 72/149] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:01 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:01.192567 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c8061-0343-453a-862d-dd55e157db8e None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:01.308255 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c8061-0343-453a-862d-dd55e157db8e None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:01.347789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf20d80-0f38-496d-9014-b7a868806126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:01.348490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf20d80-0f38-496d-9014-b7a868806126 None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:01.348769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf20d80-0f38-496d-9014-b7a868806126 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:01.349052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf20d80-0f38-496d-9014-b7a868806126 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d/users/8d6ab68412ec4430998b625c98741ff8/roles/7da1608958c94a6dbb98eaf955981d2d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:01.350970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf20d80-0f38-496d-9014-b7a868806126 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d01d0d61d804d4393d51b2d191f652d, user_id=8d6ab68412ec4430998b625c98741ff8, role_id=7da1608958c94a6dbb98eaf955981d2d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:01.352509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf20d80-0f38-496d-9014-b7a868806126 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:01.407064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ec6e8-1fcd-4afa-a60a-2458d65a665c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:01.450522 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 78/150] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:05:01 2022] PUT /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d/users/8d6ab68412ec4430998b625c98741ff8/roles/7da1608958c94a6dbb98eaf955981d2d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:05:01.548566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ec6e8-1fcd-4afa-a60a-2458d65a665c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:01.587336 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd03819-c626-4a0d-b210-97b62f32611a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:01.587914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd03819-c626-4a0d-b210-97b62f32611a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:01.588222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd03819-c626-4a0d-b210-97b62f32611a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:01.588499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd03819-c626-4a0d-b210-97b62f32611a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:01.602558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd03819-c626-4a0d-b210-97b62f32611a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-8fd03819-c626-4a0d-b210-97b62f32611a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:01.611083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:01.613318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:01.615248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:01.615248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:01.615248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:01.615248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:01.615248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:01.615248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:01.615248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:01.615248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:01.615248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:01.615248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:01.615248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:01.615248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:01.615248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 73/151] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:01.626784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf20d80-0f38-496d-9014-b7a868806126 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:01.705940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:01.756369 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf20d80-0f38-496d-9014-b7a868806126 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:01.795407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6345d-f49a-4068-964d-687b8717a770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:01.796038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6345d-f49a-4068-964d-687b8717a770 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:01.796348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6345d-f49a-4068-964d-687b8717a770 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:01.796629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6345d-f49a-4068-964d-687b8717a770 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:01.797313 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6345d-f49a-4068-964d-687b8717a770 None admin] RBAC: Adding query filter params (name=member) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:01.797720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6345d-f49a-4068-964d-687b8717a770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:01.799126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6345d-f49a-4068-964d-687b8717a770 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:01.809233 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 79/152] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:01.820518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd03819-c626-4a0d-b210-97b62f32611a None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:01.892457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:01.952632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd03819-c626-4a0d-b210-97b62f32611a None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:01.994642 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ad2ac-02a5-4577-9e37-65488baea35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:01.995284 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ad2ac-02a5-4577-9e37-65488baea35c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:01.995562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ad2ac-02a5-4577-9e37-65488baea35c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:01.995834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ad2ac-02a5-4577-9e37-65488baea35c None admin] PATH_INFO: `/v3/users/f535a214d4d940d699e82a9e017b9c1f` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:02.000991 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ad2ac-02a5-4577-9e37-65488baea35c None admin] RBAC: Authorizing `identity:get_user(user_id=f535a214d4d940d699e82a9e017b9c1f)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:02.002487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ad2ac-02a5-4577-9e37-65488baea35c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:02.004426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 74/153] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:01 2022] GET /identity/v3/users/f535a214d4d940d699e82a9e017b9c1f => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:02.010135 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6345d-f49a-4068-964d-687b8717a770 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:02.157201 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6345d-f49a-4068-964d-687b8717a770 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:02.185589 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:02.198604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad8b28-22c6-49a7-8667-5469ae7fa510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:02.199243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad8b28-22c6-49a7-8667-5469ae7fa510 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:02.199497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad8b28-22c6-49a7-8667-5469ae7fa510 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:02.199776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad8b28-22c6-49a7-8667-5469ae7fa510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:02.277393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad8b28-22c6-49a7-8667-5469ae7fa510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:02.278890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad8b28-22c6-49a7-8667-5469ae7fa510 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:02.394210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 80/154] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:02.399831 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ad2ac-02a5-4577-9e37-65488baea35c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:02.483663 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cd546-7208-4c5d-8b97-5cef74d86724 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:02.483968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cd546-7208-4c5d-8b97-5cef74d86724 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:02.484148 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cd546-7208-4c5d-8b97-5cef74d86724 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:02.487061 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 81/155] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:02.497740 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bba762-15bc-4d61-8f99-231e4c198d22 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:02.498034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bba762-15bc-4d61-8f99-231e4c198d22 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:02.498315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bba762-15bc-4d61-8f99-231e4c198d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:02.525975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ad2ac-02a5-4577-9e37-65488baea35c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:02.537551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-69bba762-15bc-4d61-8f99-231e4c198d22 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:02.538881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-69bba762-15bc-4d61-8f99-231e4c198d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:02.546597 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:02.572528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:02.574548 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5d8d7-4db3-48d5-b80d-3f8d0c7d8f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:02.575190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5d8d7-4db3-48d5-b80d-3f8d0c7d8f26 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:02.575470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5d8d7-4db3-48d5-b80d-3f8d0c7d8f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:02.575742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5d8d7-4db3-48d5-b80d-3f8d0c7d8f26 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:02.578264 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5d8d7-4db3-48d5-b80d-3f8d0c7d8f26 None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:02.579923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5d8d7-4db3-48d5-b80d-3f8d0c7d8f26 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:02.583203 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 75/156] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:02 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:02.588962 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5d8d7-4db3-48d5-b80d-3f8d0c7d8f26 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:02.689422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 82/157] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:05:02.747907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5d8d7-4db3-48d5-b80d-3f8d0c7d8f26 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:02.785736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5772b1-48f9-4cc6-ae0e-9a957ae5bfd9 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:02.786039 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5772b1-48f9-4cc6-ae0e-9a957ae5bfd9 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:02.786282 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5772b1-48f9-4cc6-ae0e-9a957ae5bfd9 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:02.787702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 83/158] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:02.793280 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038502a8-ca43-45b0-92f9-d4980311e2c5 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:02.794792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038502a8-ca43-45b0-92f9-d4980311e2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:02.795071 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038502a8-ca43-45b0-92f9-d4980311e2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:02.799287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdc122-bc53-4e1b-bd97-4c51c69293c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:02.800456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdc122-bc53-4e1b-bd97-4c51c69293c7 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:02.800731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdc122-bc53-4e1b-bd97-4c51c69293c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:02.800998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdc122-bc53-4e1b-bd97-4c51c69293c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:02.801738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdc122-bc53-4e1b-bd97-4c51c69293c7 None admin] RBAC: Adding query filter params (role.id=4f10622fcf4c4f708cd6b652d886ba3e, scope.project.id=6d01d0d61d804d4393d51b2d191f652d, user.id=f535a214d4d940d699e82a9e017b9c1f) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:02.802046 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdc122-bc53-4e1b-bd97-4c51c69293c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:02.803498 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdc122-bc53-4e1b-bd97-4c51c69293c7 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:02.822343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 76/159] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:02 2022] GET /identity/v3/role_assignments?user.id=f535a214d4d940d699e82a9e017b9c1f&scope.project.id=6d01d0d61d804d4393d51b2d191f652d&role.id=4f10622fcf4c4f708cd6b652d886ba3e => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:02.823348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-038502a8-ca43-45b0-92f9-d4980311e2c5 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:02.824583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-038502a8-ca43-45b0-92f9-d4980311e2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:02.914592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 84/160] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:05:03.118722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdc122-bc53-4e1b-bd97-4c51c69293c7 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:03.173979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:03.232461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdc122-bc53-4e1b-bd97-4c51c69293c7 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:03.271683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8846a0-4c83-41a0-9b81-9730a079dbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:03.272350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8846a0-4c83-41a0-9b81-9730a079dbda None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:03.272631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8846a0-4c83-41a0-9b81-9730a079dbda None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:03.272902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8846a0-4c83-41a0-9b81-9730a079dbda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:03.289776 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8846a0-4c83-41a0-9b81-9730a079dbda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-ab8846a0-4c83-41a0-9b81-9730a079dbda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:03.295195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:03.296150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:03.296955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:03.296955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:03.296955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:03.296955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:03.296955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:03.296955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:03.296955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:03.296955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:03.296955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:03.296955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:03.296955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:03.296955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:03.296955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 77/161] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:05:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:03.302544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038502a8-ca43-45b0-92f9-d4980311e2c5 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:03.390879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:03.411604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038502a8-ca43-45b0-92f9-d4980311e2c5 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:03.454053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d9922-1c84-49f4-ae60-6a86ef95bff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:03.454676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d9922-1c84-49f4-ae60-6a86ef95bff9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:03.454954 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d9922-1c84-49f4-ae60-6a86ef95bff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:03.455220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d9922-1c84-49f4-ae60-6a86ef95bff9 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:03.456434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d9922-1c84-49f4-ae60-6a86ef95bff9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:03.456729 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d9922-1c84-49f4-ae60-6a86ef95bff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:03.458123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d9922-1c84-49f4-ae60-6a86ef95bff9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:03.471025 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 85/162] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:03.476306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8846a0-4c83-41a0-9b81-9730a079dbda None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:03.489672 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d9922-1c84-49f4-ae60-6a86ef95bff9 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:03.626446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8846a0-4c83-41a0-9b81-9730a079dbda None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:03.634099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:03.677846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59367347-da1b-4e44-8f32-c5dadccb44b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:03.678742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59367347-da1b-4e44-8f32-c5dadccb44b3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:03.679164 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59367347-da1b-4e44-8f32-c5dadccb44b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:03.679603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59367347-da1b-4e44-8f32-c5dadccb44b3 None admin] PATH_INFO: `/v3/users/7afd1ca5c60c424cb58374006df79ec1` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:03.684843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59367347-da1b-4e44-8f32-c5dadccb44b3 None admin] RBAC: Authorizing `identity:get_user(user_id=7afd1ca5c60c424cb58374006df79ec1)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:03.686470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59367347-da1b-4e44-8f32-c5dadccb44b3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:03.688590 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 78/163] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:03 2022] GET /identity/v3/users/7afd1ca5c60c424cb58374006df79ec1 => generated 247 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:03.691131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d9922-1c84-49f4-ae60-6a86ef95bff9 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:03.714959 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9be836-0431-409e-bea4-7ba9531b6b15 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:03.715243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9be836-0431-409e-bea4-7ba9531b6b15 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:03.715589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9be836-0431-409e-bea4-7ba9531b6b15 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:03.717294 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 79/164] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:05:03.719833 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9be836-0431-409e-bea4-7ba9531b6b15 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:03.746352 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484cf6c-90ec-4899-97c8-64a51002877b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:03.747090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484cf6c-90ec-4899-97c8-64a51002877b None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:03.747279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484cf6c-90ec-4899-97c8-64a51002877b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:03.747541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484cf6c-90ec-4899-97c8-64a51002877b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:03.773893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484cf6c-90ec-4899-97c8-64a51002877b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-0484cf6c-90ec-4899-97c8-64a51002877b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:03.790164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:03.791412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:03.792252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:03.792252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:03.792252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:03.792252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:03.792252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:03.792252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:03.792252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:03.792252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:03.792252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:03.792252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:03.792252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:03.792252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:03.792252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 86/165] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:03.797884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f05431-d244-4f56-9fd4-ee94ab12cd52 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:03.798177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f05431-d244-4f56-9fd4-ee94ab12cd52 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:03.798453 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f05431-d244-4f56-9fd4-ee94ab12cd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:03.822067 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:03.845312 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-b7f05431-d244-4f56-9fd4-ee94ab12cd52 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:03.847077 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b7f05431-d244-4f56-9fd4-ee94ab12cd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:03.897253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9be836-0431-409e-bea4-7ba9531b6b15 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:03.976376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4fe8c-2af1-4b25-84b7-4ab56371b6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:03.976938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4fe8c-2af1-4b25-84b7-4ab56371b6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:03.977223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4fe8c-2af1-4b25-84b7-4ab56371b6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:03.977502 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4fe8c-2af1-4b25-84b7-4ab56371b6f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:03.978070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 87/166] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:03.978994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f05431-d244-4f56-9fd4-ee94ab12cd52 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:04.089542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:04.125233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4fe8c-2af1-4b25-84b7-4ab56371b6f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:04.126736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4fe8c-2af1-4b25-84b7-4ab56371b6f1 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:04.152032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f05431-d244-4f56-9fd4-ee94ab12cd52 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:04.201911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd242c27-6ec9-4add-a6ae-97d73a2e4999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:04.202549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd242c27-6ec9-4add-a6ae-97d73a2e4999 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:04.202832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd242c27-6ec9-4add-a6ae-97d73a2e4999 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:04.203144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd242c27-6ec9-4add-a6ae-97d73a2e4999 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:04.203979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd242c27-6ec9-4add-a6ae-97d73a2e4999 None admin] RBAC: Adding query filter params (name=member) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:04.204314 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd242c27-6ec9-4add-a6ae-97d73a2e4999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:04.206267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd242c27-6ec9-4add-a6ae-97d73a2e4999 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:04.207832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:04.219970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 88/167] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:04.226242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd242c27-6ec9-4add-a6ae-97d73a2e4999 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:04.288054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 80/168] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:04.298337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4fe8c-2af1-4b25-84b7-4ab56371b6f1 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:04.429343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd242c27-6ec9-4add-a6ae-97d73a2e4999 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:04.434834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:04.479580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4fe8c-2af1-4b25-84b7-4ab56371b6f1 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:04.537280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7a34f-865c-4ef9-a87e-e96cc69c896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:04.538424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04034c39-3a60-4e81-a998-017de88290c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:04.538700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7a34f-865c-4ef9-a87e-e96cc69c896a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:04.539016 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04034c39-3a60-4e81-a998-017de88290c0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:04.539304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04034c39-3a60-4e81-a998-017de88290c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:04.539573 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04034c39-3a60-4e81-a998-017de88290c0 None admin] PATH_INFO: `/v3/users/f535a214d4d940d699e82a9e017b9c1f` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:04.539872 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7a34f-865c-4ef9-a87e-e96cc69c896a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:04.540454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7a34f-865c-4ef9-a87e-e96cc69c896a None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:04.543888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7a34f-865c-4ef9-a87e-e96cc69c896a None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:04.545114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04034c39-3a60-4e81-a998-017de88290c0 None admin] RBAC: Authorizing `identity:get_user(user_id=f535a214d4d940d699e82a9e017b9c1f)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:04.545988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7a34f-865c-4ef9-a87e-e96cc69c896a None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:04.546599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04034c39-3a60-4e81-a998-017de88290c0 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:04.548158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 81/169] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:04 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:04.549187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 89/170] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:04 2022] GET /identity/v3/users/f535a214d4d940d699e82a9e017b9c1f => generated 275 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:04.555312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7a34f-865c-4ef9-a87e-e96cc69c896a None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:04.556296 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04034c39-3a60-4e81-a998-017de88290c0 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:04.697436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:04.782773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04034c39-3a60-4e81-a998-017de88290c0 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:04.804060 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7a34f-865c-4ef9-a87e-e96cc69c896a None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:04.866264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293f6cd-3915-4009-a972-e3734b613f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:04.866895 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293f6cd-3915-4009-a972-e3734b613f79 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:04.867183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293f6cd-3915-4009-a972-e3734b613f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:04.867459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293f6cd-3915-4009-a972-e3734b613f79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:04.887531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99003752-45b5-4e1e-ad8b-201c29b5989c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:04.888158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99003752-45b5-4e1e-ad8b-201c29b5989c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:04.888464 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99003752-45b5-4e1e-ad8b-201c29b5989c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:04.888757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99003752-45b5-4e1e-ad8b-201c29b5989c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:04.889486 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99003752-45b5-4e1e-ad8b-201c29b5989c None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=6d01d0d61d804d4393d51b2d191f652d, user.id=7afd1ca5c60c424cb58374006df79ec1) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:04.889791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99003752-45b5-4e1e-ad8b-201c29b5989c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:04.891222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99003752-45b5-4e1e-ad8b-201c29b5989c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:04.905027 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 82/171] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:04 2022] GET /identity/v3/role_assignments?user.id=7afd1ca5c60c424cb58374006df79ec1&scope.project.id=6d01d0d61d804d4393d51b2d191f652d&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 608 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:04.905912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99003752-45b5-4e1e-ad8b-201c29b5989c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:04.923470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:05.029395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293f6cd-3915-4009-a972-e3734b613f79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:05.030916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293f6cd-3915-4009-a972-e3734b613f79 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:05.159380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99003752-45b5-4e1e-ad8b-201c29b5989c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:05.184552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:05.226349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9e447-0fb8-4c1c-8655-5f87730c11f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:05.226999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9e447-0fb8-4c1c-8655-5f87730c11f0 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:05.227269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9e447-0fb8-4c1c-8655-5f87730c11f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:05.227534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9e447-0fb8-4c1c-8655-5f87730c11f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:05.245952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde9e447-0fb8-4c1c-8655-5f87730c11f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:05.247693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 90/172] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:05.248986 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad9811-bb43-44c0-bdcd-06bacfd40c7e None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:05.249275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad9811-bb43-44c0-bdcd-06bacfd40c7e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:05.249544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad9811-bb43-44c0-bdcd-06bacfd40c7e None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:05.251080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 91/173] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:05.252112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8586b18-f63a-488c-8b3a-50fbcc1583a8 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:05.252440 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8586b18-f63a-488c-8b3a-50fbcc1583a8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:05.252730 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8586b18-f63a-488c-8b3a-50fbcc1583a8 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-dde9e447-0fb8-4c1c-8655-5f87730c11f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:05.256191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:05.257088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:05.257957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:05.257957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:05.257957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:05.257957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:05.257957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:05.257957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:05.257957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:05.257957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:05.257957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:05.257957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:05.257957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:05.257957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:05.258561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 92/174] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:05.258561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8586b18-f63a-488c-8b3a-50fbcc1583a8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:05.264429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 83/175] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:05:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:05.264429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80aa19a-e44d-47cb-9855-2ec6b3da5c70 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:05.264429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80aa19a-e44d-47cb-9855-2ec6b3da5c70 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:05.264429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80aa19a-e44d-47cb-9855-2ec6b3da5c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:05.292705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b80aa19a-e44d-47cb-9855-2ec6b3da5c70 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:05.293911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b80aa19a-e44d-47cb-9855-2ec6b3da5c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:05.374166 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:05.375530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8586b18-f63a-488c-8b3a-50fbcc1583a8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:05.383161 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 84/176] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:05.385044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83ac61-beca-4a47-88c6-a6d0966f0429 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:05.385325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83ac61-beca-4a47-88c6-a6d0966f0429 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:05.389769 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83ac61-beca-4a47-88c6-a6d0966f0429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:05.413963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-1e83ac61-beca-4a47-88c6-a6d0966f0429 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:05.415786 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-1e83ac61-beca-4a47-88c6-a6d0966f0429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:05.428913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f269e-2469-4a13-b31e-b308c7eed19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:05.429570 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f269e-2469-4a13-b31e-b308c7eed19d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:05.430106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f269e-2469-4a13-b31e-b308c7eed19d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:05.430106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f269e-2469-4a13-b31e-b308c7eed19d None admin] PATH_INFO: `/v3/projects/45de5144b71346cca2ca4f287a2ba098` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:05.437002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f269e-2469-4a13-b31e-b308c7eed19d None admin] RBAC: Authorizing `identity:get_project(project_id=45de5144b71346cca2ca4f287a2ba098)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:05.438526 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f269e-2469-4a13-b31e-b308c7eed19d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:05.441097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 93/177] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:05 2022] GET /identity/v3/projects/45de5144b71346cca2ca4f287a2ba098 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:05.441950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f269e-2469-4a13-b31e-b308c7eed19d None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:05.497748 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 85/178] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:05.498611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83ac61-beca-4a47-88c6-a6d0966f0429 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:05.553084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:05.669067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83ac61-beca-4a47-88c6-a6d0966f0429 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:05.744534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:05.752127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f269e-2469-4a13-b31e-b308c7eed19d None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:05.767593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f0dbd-fa69-44bb-b0ac-138b0cd7feb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:05.768937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f0dbd-fa69-44bb-b0ac-138b0cd7feb5 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:05.769228 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f0dbd-fa69-44bb-b0ac-138b0cd7feb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:05.769505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f0dbd-fa69-44bb-b0ac-138b0cd7feb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:05.770252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f0dbd-fa69-44bb-b0ac-138b0cd7feb5 None admin] RBAC: Adding query filter params (role.id=4f10622fcf4c4f708cd6b652d886ba3e, scope.project.id=45de5144b71346cca2ca4f287a2ba098, user.id=f535a214d4d940d699e82a9e017b9c1f) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:05.770560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f0dbd-fa69-44bb-b0ac-138b0cd7feb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:05.772024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f0dbd-fa69-44bb-b0ac-138b0cd7feb5 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:05.783084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 86/179] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:05 2022] GET /identity/v3/role_assignments?user.id=f535a214d4d940d699e82a9e017b9c1f&scope.project.id=45de5144b71346cca2ca4f287a2ba098&role.id=4f10622fcf4c4f708cd6b652d886ba3e => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:05.784518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f0dbd-fa69-44bb-b0ac-138b0cd7feb5 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:05.852931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ac5dc-c039-4659-b3d3-f466728d231c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:05.853563 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ac5dc-c039-4659-b3d3-f466728d231c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:05.853842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ac5dc-c039-4659-b3d3-f466728d231c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:05.854113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ac5dc-c039-4659-b3d3-f466728d231c None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:05.854789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ac5dc-c039-4659-b3d3-f466728d231c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:05.855097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ac5dc-c039-4659-b3d3-f466728d231c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:05.860113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ac5dc-c039-4659-b3d3-f466728d231c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:05.871243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 94/180] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:05.873087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde970e-b7d9-427c-a78b-c8f707029c94 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:05.873633 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde970e-b7d9-427c-a78b-c8f707029c94 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:05.874115 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde970e-b7d9-427c-a78b-c8f707029c94 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:05.880479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 95/181] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:05.881865 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde970e-b7d9-427c-a78b-c8f707029c94 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:05.929191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f0dbd-fa69-44bb-b0ac-138b0cd7feb5 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:05.975928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd12b32-6b27-4522-9510-24abc111a9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:05.976760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd12b32-6b27-4522-9510-24abc111a9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:05.977041 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd12b32-6b27-4522-9510-24abc111a9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:05.977309 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd12b32-6b27-4522-9510-24abc111a9c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:05.988661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd12b32-6b27-4522-9510-24abc111a9c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:05.991803 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-8dd12b32-6b27-4522-9510-24abc111a9c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:05.994482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:05.995546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:05.996581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:05.996581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:05.996581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:05.996581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:05.996581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:05.996581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:05.996581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:05.996581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:05.996581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:05.996581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:05.996581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:05.996581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:05.998844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 87/182] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:05:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:05.999850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd12b32-6b27-4522-9510-24abc111a9c4 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:06.001009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde970e-b7d9-427c-a78b-c8f707029c94 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:06.042116 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28685dc5-4783-4bef-adf3-5490ff62e0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:06.042756 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28685dc5-4783-4bef-adf3-5490ff62e0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:06.043030 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28685dc5-4783-4bef-adf3-5490ff62e0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:06.043334 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28685dc5-4783-4bef-adf3-5490ff62e0c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:06.057697 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28685dc5-4783-4bef-adf3-5490ff62e0c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:06.060575 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-28685dc5-4783-4bef-adf3-5490ff62e0c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:06.062683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:06.063612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:06.064444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:06.064444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:06.064444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:06.064444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:06.064444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:06.064444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:06.064444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:06.064444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:06.064444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:06.064444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:06.064444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:06.064444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:06.064444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 96/183] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:06.065635 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46650c4d-d4a2-41e2-a6b7-9a70ff93a8a9 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:06.065786 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46650c4d-d4a2-41e2-a6b7-9a70ff93a8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:06.066056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46650c4d-d4a2-41e2-a6b7-9a70ff93a8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:06.084982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-46650c4d-d4a2-41e2-a6b7-9a70ff93a8a9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:06.086168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-46650c4d-d4a2-41e2-a6b7-9a70ff93a8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:06.112275 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd12b32-6b27-4522-9510-24abc111a9c4 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:06.146189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf5c59-7750-4596-99c9-c94bd16f909a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:06.146816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf5c59-7750-4596-99c9-c94bd16f909a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:06.147104 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf5c59-7750-4596-99c9-c94bd16f909a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:06.147380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf5c59-7750-4596-99c9-c94bd16f909a None admin] PATH_INFO: `/v3/users/f535a214d4d940d699e82a9e017b9c1f` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:06.148535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf5c59-7750-4596-99c9-c94bd16f909a None admin] RBAC: Authorizing `identity:get_user(user_id=f535a214d4d940d699e82a9e017b9c1f)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:06.149974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf5c59-7750-4596-99c9-c94bd16f909a None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:06.151820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 88/184] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:05 2022] GET /identity/v3/users/f535a214d4d940d699e82a9e017b9c1f => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:06.152591 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 97/185] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:06.153267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46650c4d-d4a2-41e2-a6b7-9a70ff93a8a9 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:06.156910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e274de-826b-40a0-ad3e-7f95f7e61ba2 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:06.157103 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e274de-826b-40a0-ad3e-7f95f7e61ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:06.157389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e274de-826b-40a0-ad3e-7f95f7e61ba2 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:06.158845 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 89/186] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:06.159418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e274de-826b-40a0-ad3e-7f95f7e61ba2 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:06.249631 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:06.265439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46650c4d-d4a2-41e2-a6b7-9a70ff93a8a9 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:06.273463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e274de-826b-40a0-ad3e-7f95f7e61ba2 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:06.291958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:06.301490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a5576-da9e-4249-bd89-11f9d9aee159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:06.302769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a5576-da9e-4249-bd89-11f9d9aee159 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:06.303791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a5576-da9e-4249-bd89-11f9d9aee159 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:06.304060 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a5576-da9e-4249-bd89-11f9d9aee159 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:06.304777 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a5576-da9e-4249-bd89-11f9d9aee159 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:06.305075 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a5576-da9e-4249-bd89-11f9d9aee159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:06.306443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a5576-da9e-4249-bd89-11f9d9aee159 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:06.313219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474ede3-62dd-49df-80f1-08bf4c191386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:06.314176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474ede3-62dd-49df-80f1-08bf4c191386 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:06.314634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474ede3-62dd-49df-80f1-08bf4c191386 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:06.314751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 98/187] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:06.315702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474ede3-62dd-49df-80f1-08bf4c191386 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:06.315963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a5576-da9e-4249-bd89-11f9d9aee159 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:06.316865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474ede3-62dd-49df-80f1-08bf4c191386 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:06.317173 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474ede3-62dd-49df-80f1-08bf4c191386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:06.318569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474ede3-62dd-49df-80f1-08bf4c191386 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:06.329944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 90/188] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:06.331632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5d5ea-d12b-4d44-86ec-c5131ea83fe3 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:06.331920 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5d5ea-d12b-4d44-86ec-c5131ea83fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:06.332645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5d5ea-d12b-4d44-86ec-c5131ea83fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:06.356715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-33a5d5ea-d12b-4d44-86ec-c5131ea83fe3 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:06.358080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-33a5d5ea-d12b-4d44-86ec-c5131ea83fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:06.426576 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a5576-da9e-4249-bd89-11f9d9aee159 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:06.429478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 91/189] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:06.430323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5d5ea-d12b-4d44-86ec-c5131ea83fe3 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:06.463022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262fcc6-f5cd-4adb-b877-a5ed8ac1af88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:06.464441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262fcc6-f5cd-4adb-b877-a5ed8ac1af88 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:06.464926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262fcc6-f5cd-4adb-b877-a5ed8ac1af88 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:06.465406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262fcc6-f5cd-4adb-b877-a5ed8ac1af88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:06.485932 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:06.541297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5d5ea-d12b-4d44-86ec-c5131ea83fe3 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:06.541922 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262fcc6-f5cd-4adb-b877-a5ed8ac1af88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:06.542836 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262fcc6-f5cd-4adb-b877-a5ed8ac1af88 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:06.553055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:06.576958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69891be1-63eb-4fce-b97a-61f5257f71a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:06.577596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891be1-63eb-4fce-b97a-61f5257f71a6 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:06.577878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891be1-63eb-4fce-b97a-61f5257f71a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:06.578150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891be1-63eb-4fce-b97a-61f5257f71a6 None admin] PATH_INFO: `/v3/users/824da08c89904f77aec01acb63e9cf08` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:06.592782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69891be1-63eb-4fce-b97a-61f5257f71a6 None admin] RBAC: Authorizing `identity:get_user(user_id=824da08c89904f77aec01acb63e9cf08)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:06.594332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69891be1-63eb-4fce-b97a-61f5257f71a6 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:06.596023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 92/190] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:06 2022] GET /identity/v3/users/824da08c89904f77aec01acb63e9cf08 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:06.596847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69891be1-63eb-4fce-b97a-61f5257f71a6 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:06.667450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 99/191] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:06.668262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262fcc6-f5cd-4adb-b877-a5ed8ac1af88 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:06.700567 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69891be1-63eb-4fce-b97a-61f5257f71a6 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:06.715903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:06.735717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2220392-28cd-4492-82e7-4fc008034512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:06.736385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220392-28cd-4492-82e7-4fc008034512 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:06.736768 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220392-28cd-4492-82e7-4fc008034512 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:06.737043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220392-28cd-4492-82e7-4fc008034512 None admin] PATH_INFO: `/v3/users/8d6ab68412ec4430998b625c98741ff8` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:06.738183 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2220392-28cd-4492-82e7-4fc008034512 None admin] RBAC: Authorizing `identity:get_user(user_id=8d6ab68412ec4430998b625c98741ff8)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:06.739613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2220392-28cd-4492-82e7-4fc008034512 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:06.741228 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 93/192] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:06 2022] GET /identity/v3/users/8d6ab68412ec4430998b625c98741ff8 => generated 289 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:06.743147 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2220392-28cd-4492-82e7-4fc008034512 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:06.769681 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262fcc6-f5cd-4adb-b877-a5ed8ac1af88 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:06.778489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:06.803700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c736096-b4bd-4e9f-8d68-03d7ed2ec90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:06.804273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c736096-b4bd-4e9f-8d68-03d7ed2ec90e None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:06.804557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c736096-b4bd-4e9f-8d68-03d7ed2ec90e None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:06.804830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c736096-b4bd-4e9f-8d68-03d7ed2ec90e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:06.806021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c736096-b4bd-4e9f-8d68-03d7ed2ec90e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:06.807509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c736096-b4bd-4e9f-8d68-03d7ed2ec90e None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:06.809286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 100/193] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:06.809962 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c736096-b4bd-4e9f-8d68-03d7ed2ec90e None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:06.844999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2220392-28cd-4492-82e7-4fc008034512 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:06.878547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8ae68-e4e2-49b1-98d5-2fc61bef3828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:06.879146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8ae68-e4e2-49b1-98d5-2fc61bef3828 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:06.879427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8ae68-e4e2-49b1-98d5-2fc61bef3828 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:06.879702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8ae68-e4e2-49b1-98d5-2fc61bef3828 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:06.910268 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c736096-b4bd-4e9f-8d68-03d7ed2ec90e None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:06.923869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:06.944239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0413489-f3c7-4098-8c22-9f52077b4c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:06.944874 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0413489-f3c7-4098-8c22-9f52077b4c17 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:06.945154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0413489-f3c7-4098-8c22-9f52077b4c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:06.945464 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0413489-f3c7-4098-8c22-9f52077b4c17 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:06.946642 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0413489-f3c7-4098-8c22-9f52077b4c17 None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:06.946757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8ae68-e4e2-49b1-98d5-2fc61bef3828 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:06.948087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0413489-f3c7-4098-8c22-9f52077b4c17 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:06.948227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8ae68-e4e2-49b1-98d5-2fc61bef3828 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:06.949928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 101/194] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:06 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:06.953010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0413489-f3c7-4098-8c22-9f52077b4c17 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:06.981860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:07.046795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 94/195] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:07.047641 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8ae68-e4e2-49b1-98d5-2fc61bef3828 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:07.053275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0413489-f3c7-4098-8c22-9f52077b4c17 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:07.086331 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e3e82-2572-4e9c-b650-eb983dda5272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:07.086879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e3e82-2572-4e9c-b650-eb983dda5272 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:07.087156 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e3e82-2572-4e9c-b650-eb983dda5272 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:07.087426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e3e82-2572-4e9c-b650-eb983dda5272 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:07.098234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e3e82-2572-4e9c-b650-eb983dda5272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-e73e3e82-2572-4e9c-b650-eb983dda5272 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:07.103618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:07.104654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:07.105457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:07.105457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:07.105457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:07.105457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:07.105457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:07.105457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:07.105457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:07.105457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:07.105457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:07.105457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:07.105457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:07.105457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:07.105457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 102/196] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:05:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:07.106118 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e3e82-2572-4e9c-b650-eb983dda5272 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:07.148767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:07.150779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8ae68-e4e2-49b1-98d5-2fc61bef3828 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:07.185576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05136894-0618-427a-b7eb-42fae67c6339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:07.186181 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05136894-0618-427a-b7eb-42fae67c6339 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:07.186463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05136894-0618-427a-b7eb-42fae67c6339 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:07.186734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05136894-0618-427a-b7eb-42fae67c6339 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:07.190327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:07.206162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e3e82-2572-4e9c-b650-eb983dda5272 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:07.239754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50828a33-e9cf-4d51-bbac-0be951c65aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:07.240396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50828a33-e9cf-4d51-bbac-0be951c65aa5 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:07.240685 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50828a33-e9cf-4d51-bbac-0be951c65aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:07.240960 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50828a33-e9cf-4d51-bbac-0be951c65aa5 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:07.241609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50828a33-e9cf-4d51-bbac-0be951c65aa5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:07.243055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50828a33-e9cf-4d51-bbac-0be951c65aa5 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:07.255299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05136894-0618-427a-b7eb-42fae67c6339 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:07.256814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05136894-0618-427a-b7eb-42fae67c6339 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:07.351909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 95/197] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:07.352732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05136894-0618-427a-b7eb-42fae67c6339 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:07.375628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 103/198] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:05:07 2022] POST /identity/v3/users => generated 297 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:05:07.388377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50828a33-e9cf-4d51-bbac-0be951c65aa5 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:07.389707 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:07.465153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05136894-0618-427a-b7eb-42fae67c6339 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:07.484059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:07.488425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50828a33-e9cf-4d51-bbac-0be951c65aa5 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:07.505161 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6f5e4-f55e-44fa-aed3-dab6e4b74ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:07.505844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6f5e4-f55e-44fa-aed3-dab6e4b74ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:07.506068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6f5e4-f55e-44fa-aed3-dab6e4b74ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:07.506345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6f5e4-f55e-44fa-aed3-dab6e4b74ef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:07.507091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6f5e4-f55e-44fa-aed3-dab6e4b74ef5 None admin] RBAC: Adding query filter params (role.id=f1d8af7d158e4e02a6e77492178db259, scope.project.id=6d01d0d61d804d4393d51b2d191f652d, user.id=f535a214d4d940d699e82a9e017b9c1f) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:07.507426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6f5e4-f55e-44fa-aed3-dab6e4b74ef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:07.509812 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6f5e4-f55e-44fa-aed3-dab6e4b74ef5 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:07.517607 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 96/199] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:07 2022] GET /identity/v3/role_assignments?user.id=f535a214d4d940d699e82a9e017b9c1f&scope.project.id=6d01d0d61d804d4393d51b2d191f652d&role.id=f1d8af7d158e4e02a6e77492178db259 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:07.519074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6f5e4-f55e-44fa-aed3-dab6e4b74ef5 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:07.542683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304518b6-8e94-4d99-bc5d-8905920a8cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:07.543349 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304518b6-8e94-4d99-bc5d-8905920a8cec None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:07.543628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304518b6-8e94-4d99-bc5d-8905920a8cec None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:07.546674 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304518b6-8e94-4d99-bc5d-8905920a8cec None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:07.546674 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304518b6-8e94-4d99-bc5d-8905920a8cec None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:07.546674 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304518b6-8e94-4d99-bc5d-8905920a8cec None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:07.548638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 104/200] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:07 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:07.549820 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304518b6-8e94-4d99-bc5d-8905920a8cec None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:07.665628 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:07.669543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6f5e4-f55e-44fa-aed3-dab6e4b74ef5 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:07.710979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304518b6-8e94-4d99-bc5d-8905920a8cec None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:07.724143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06699e31-a71c-4d26-8857-e4f4b1a71b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:07.724868 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06699e31-a71c-4d26-8857-e4f4b1a71b9b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:07.725146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06699e31-a71c-4d26-8857-e4f4b1a71b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:07.725422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06699e31-a71c-4d26-8857-e4f4b1a71b9b None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:07.726098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06699e31-a71c-4d26-8857-e4f4b1a71b9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:07.726408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06699e31-a71c-4d26-8857-e4f4b1a71b9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:07.727797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06699e31-a71c-4d26-8857-e4f4b1a71b9b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:07.740933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 97/201] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:07.741795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06699e31-a71c-4d26-8857-e4f4b1a71b9b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:07.766573 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae36d34-7b4e-4e0f-baae-d82de8e916b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:07.767465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae36d34-7b4e-4e0f-baae-d82de8e916b5 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:07.767745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae36d34-7b4e-4e0f-baae-d82de8e916b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:07.768068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae36d34-7b4e-4e0f-baae-d82de8e916b5 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:07.770021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae36d34-7b4e-4e0f-baae-d82de8e916b5 None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:07.771664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae36d34-7b4e-4e0f-baae-d82de8e916b5 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:07.773892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 105/202] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:07 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:07.776966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae36d34-7b4e-4e0f-baae-d82de8e916b5 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:07.820537 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:07.892253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06699e31-a71c-4d26-8857-e4f4b1a71b9b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:07.895739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae36d34-7b4e-4e0f-baae-d82de8e916b5 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:07.934400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad821b-e547-4bf6-ae26-d58907a27d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:07.935158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad821b-e547-4bf6-ae26-d58907a27d60 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:07.935438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad821b-e547-4bf6-ae26-d58907a27d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:07.935710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad821b-e547-4bf6-ae26-d58907a27d60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:07.936470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad821b-e547-4bf6-ae26-d58907a27d60 None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24, user.id=824da08c89904f77aec01acb63e9cf08) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:07.936777 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad821b-e547-4bf6-ae26-d58907a27d60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:07.937917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf001e-8b3c-4ae3-9263-52bd56af3d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:07.938231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad821b-e547-4bf6-ae26-d58907a27d60 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:07.938997 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf001e-8b3c-4ae3-9263-52bd56af3d5d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:07.939479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf001e-8b3c-4ae3-9263-52bd56af3d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:07.940024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf001e-8b3c-4ae3-9263-52bd56af3d5d None admin] PATH_INFO: `/v3/users/824da08c89904f77aec01acb63e9cf08` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:07.941448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf001e-8b3c-4ae3-9263-52bd56af3d5d None admin] RBAC: Authorizing `identity:get_user(user_id=824da08c89904f77aec01acb63e9cf08)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:07.943109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf001e-8b3c-4ae3-9263-52bd56af3d5d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:07.944058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 98/203] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:07 2022] GET /identity/v3/role_assignments?user.id=824da08c89904f77aec01acb63e9cf08&scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:07.945054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad821b-e547-4bf6-ae26-d58907a27d60 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:07.945743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 106/204] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:07 2022] GET /identity/v3/users/824da08c89904f77aec01acb63e9cf08 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:07.955374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf001e-8b3c-4ae3-9263-52bd56af3d5d None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:07.978946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:08.075057 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad821b-e547-4bf6-ae26-d58907a27d60 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:08.127616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddeecac-37f5-4f79-840d-5cc8f336a151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:08.128250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddeecac-37f5-4f79-840d-5cc8f336a151 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:08.128642 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddeecac-37f5-4f79-840d-5cc8f336a151 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:08.128915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddeecac-37f5-4f79-840d-5cc8f336a151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:08.129768 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddeecac-37f5-4f79-840d-5cc8f336a151 None admin] RBAC: Adding query filter params (role.id=7da1608958c94a6dbb98eaf955981d2d, scope.project.id=6d01d0d61d804d4393d51b2d191f652d, user.id=8d6ab68412ec4430998b625c98741ff8) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:08.130074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddeecac-37f5-4f79-840d-5cc8f336a151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:08.131522 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddeecac-37f5-4f79-840d-5cc8f336a151 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:08.138001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 99/205] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:07 2022] GET /identity/v3/role_assignments?user.id=8d6ab68412ec4430998b625c98741ff8&scope.project.id=6d01d0d61d804d4393d51b2d191f652d&role.id=7da1608958c94a6dbb98eaf955981d2d => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:08.147366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf001e-8b3c-4ae3-9263-52bd56af3d5d None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:08.149142 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:08.200030 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdeb553-9c22-4ef0-9230-6ffc9d692abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:08.200760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdeb553-9c22-4ef0-9230-6ffc9d692abf None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:08.201042 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdeb553-9c22-4ef0-9230-6ffc9d692abf None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:08.201310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdeb553-9c22-4ef0-9230-6ffc9d692abf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:08.298248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdeb553-9c22-4ef0-9230-6ffc9d692abf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:08.300112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdeb553-9c22-4ef0-9230-6ffc9d692abf None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:08.426846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 107/206] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:05:08.433351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddeecac-37f5-4f79-840d-5cc8f336a151 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:08.542055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:08.551940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddeecac-37f5-4f79-840d-5cc8f336a151 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:08.587408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e408bf5-b0ad-433e-aa38-9f22c1b21f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:08.588037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e408bf5-b0ad-433e-aa38-9f22c1b21f6c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:08.588370 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e408bf5-b0ad-433e-aa38-9f22c1b21f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:08.588649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e408bf5-b0ad-433e-aa38-9f22c1b21f6c None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:08.589809 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e408bf5-b0ad-433e-aa38-9f22c1b21f6c None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:08.591259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e408bf5-b0ad-433e-aa38-9f22c1b21f6c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:08.593148 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 100/207] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:08 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:08.599390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdeb553-9c22-4ef0-9230-6ffc9d692abf None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:08.611059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:08.726577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdeb553-9c22-4ef0-9230-6ffc9d692abf None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:08.761466 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e22c2-6a33-47e3-9f4a-52fe2e7879d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:08.762075 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e22c2-6a33-47e3-9f4a-52fe2e7879d9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:08.762353 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e22c2-6a33-47e3-9f4a-52fe2e7879d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:08.762622 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e22c2-6a33-47e3-9f4a-52fe2e7879d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:08.763352 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e22c2-6a33-47e3-9f4a-52fe2e7879d9 None admin] RBAC: Adding query filter params (role.id=f1d8af7d158e4e02a6e77492178db259, scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24, user.id=824da08c89904f77aec01acb63e9cf08) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:08.763654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e22c2-6a33-47e3-9f4a-52fe2e7879d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:08.765129 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e22c2-6a33-47e3-9f4a-52fe2e7879d9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:08.771615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 108/208] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:08 2022] GET /identity/v3/role_assignments?user.id=824da08c89904f77aec01acb63e9cf08&scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24&role.id=f1d8af7d158e4e02a6e77492178db259 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:09.697803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d688f0-3a20-4574-bad5-5821849d94be None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:09.698331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d688f0-3a20-4574-bad5-5821849d94be None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:09.698331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d688f0-3a20-4574-bad5-5821849d94be None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:09.699675 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 101/209] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:09.705107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ff5d9-18c3-4568-b1b8-284696753725 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:09.706086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ff5d9-18c3-4568-b1b8-284696753725 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:09.706194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ff5d9-18c3-4568-b1b8-284696753725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:09.725596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-887ff5d9-18c3-4568-b1b8-284696753725 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:09.726797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-887ff5d9-18c3-4568-b1b8-284696753725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:09.778290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c655fe2-950f-4c98-8052-14d4aaa23c8e None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:09.778613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c655fe2-950f-4c98-8052-14d4aaa23c8e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:09.778849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c655fe2-950f-4c98-8052-14d4aaa23c8e None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:09.780615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 102/210] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:09.785986 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f13b30-61db-4185-b53f-c0a5394e862a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:09.786278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f13b30-61db-4185-b53f-c0a5394e862a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:09.786557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f13b30-61db-4185-b53f-c0a5394e862a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:09.800807 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:09.807188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-45f13b30-61db-4185-b53f-c0a5394e862a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:09.812055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 109/211] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:09.812357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-45f13b30-61db-4185-b53f-c0a5394e862a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:09.883823 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 103/212] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:05:10.059855 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4250d-5d95-479b-9bf4-e4c27e8dc1d2 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:10.060242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4250d-5d95-479b-9bf4-e4c27e8dc1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:10.060481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4250d-5d95-479b-9bf4-e4c27e8dc1d2 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:10.062093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 110/213] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:10.069835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4250d-5d95-479b-9bf4-e4c27e8dc1d2 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:10.070290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152e3d6-2d71-46a5-80dc-d23b5228ef38 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:10.070751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152e3d6-2d71-46a5-80dc-d23b5228ef38 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:10.071026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152e3d6-2d71-46a5-80dc-d23b5228ef38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:10.095195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d152e3d6-2d71-46a5-80dc-d23b5228ef38 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:10.097269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d152e3d6-2d71-46a5-80dc-d23b5228ef38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:10.180283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 104/214] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:05:10.186436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4250d-5d95-479b-9bf4-e4c27e8dc1d2 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:10.221874 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd269a9a-51da-40cd-8259-9f8773563e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:10.222832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd269a9a-51da-40cd-8259-9f8773563e70 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:10.223334 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd269a9a-51da-40cd-8259-9f8773563e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:10.223808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd269a9a-51da-40cd-8259-9f8773563e70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:10.234377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd269a9a-51da-40cd-8259-9f8773563e70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-cd269a9a-51da-40cd-8259-9f8773563e70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:10.239477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:10.240367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:10.241141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:10.241141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:10.241141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:10.241141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:10.241141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:10.241141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:10.241141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:10.241141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:10.241141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:10.241141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:10.241141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:10.241141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:10.243330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 111/215] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:10.249467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152e3d6-2d71-46a5-80dc-d23b5228ef38 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:10.253704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd269a9a-51da-40cd-8259-9f8773563e70 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:10.275163 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:10.314935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:10.365225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152e3d6-2d71-46a5-80dc-d23b5228ef38 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:10.365808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd269a9a-51da-40cd-8259-9f8773563e70 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:10.401610 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5820f-944c-427a-b30d-5a5a4a6c1fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:10.402222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5820f-944c-427a-b30d-5a5a4a6c1fdf None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:10.402500 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5820f-944c-427a-b30d-5a5a4a6c1fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:10.402776 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5820f-944c-427a-b30d-5a5a4a6c1fdf None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:10.403461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5820f-944c-427a-b30d-5a5a4a6c1fdf None admin] RBAC: Adding query filter params (name=member) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:10.403743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5820f-944c-427a-b30d-5a5a4a6c1fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:10.405151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5820f-944c-427a-b30d-5a5a4a6c1fdf None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:10.409942 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808bd79-fef6-4f44-8cac-d5b1dc09b051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:10.411272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808bd79-fef6-4f44-8cac-d5b1dc09b051 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:10.412042 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808bd79-fef6-4f44-8cac-d5b1dc09b051 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:10.412535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808bd79-fef6-4f44-8cac-d5b1dc09b051 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:10.418122 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 105/216] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:10.418249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808bd79-fef6-4f44-8cac-d5b1dc09b051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:10.418249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808bd79-fef6-4f44-8cac-d5b1dc09b051 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:10.419660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 112/217] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:10.419978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5820f-944c-427a-b30d-5a5a4a6c1fdf None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:10.428203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808bd79-fef6-4f44-8cac-d5b1dc09b051 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:10.519575 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:10.540510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5820f-944c-427a-b30d-5a5a4a6c1fdf None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:10.543562 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808bd79-fef6-4f44-8cac-d5b1dc09b051 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:10.570610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:10.577721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83816d-a148-4d39-a8d7-4eccf35d02ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:10.588373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83816d-a148-4d39-a8d7-4eccf35d02ea None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:10.588373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83816d-a148-4d39-a8d7-4eccf35d02ea None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:10.588373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83816d-a148-4d39-a8d7-4eccf35d02ea None admin] PATH_INFO: `/v3/users/4ff0650e866b4768a49d211618c7fbd2` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:10.588625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d3fef-38a7-4d63-9637-c30b4bc91366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:10.588625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d3fef-38a7-4d63-9637-c30b4bc91366 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:10.588625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d3fef-38a7-4d63-9637-c30b4bc91366 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:10.588625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d3fef-38a7-4d63-9637-c30b4bc91366 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:10.588874 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d3fef-38a7-4d63-9637-c30b4bc91366 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:10.588874 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d3fef-38a7-4d63-9637-c30b4bc91366 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:10.594341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83816d-a148-4d39-a8d7-4eccf35d02ea None admin] RBAC: Authorizing `identity:get_user(user_id=4ff0650e866b4768a49d211618c7fbd2)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:10.595463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83816d-a148-4d39-a8d7-4eccf35d02ea None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:10.602571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 106/218] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:10 2022] GET /identity/v3/users/4ff0650e866b4768a49d211618c7fbd2 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:10.602571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83816d-a148-4d39-a8d7-4eccf35d02ea None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:10.707137 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83816d-a148-4d39-a8d7-4eccf35d02ea None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:10.718393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 113/219] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:05:10 2022] POST /identity/v3/users => generated 297 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:05:10.719755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d3fef-38a7-4d63-9637-c30b4bc91366 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:10.755829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9334ca-ba1a-4566-a925-dc686f67879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:10.756516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9334ca-ba1a-4566-a925-dc686f67879a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:10.756788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9334ca-ba1a-4566-a925-dc686f67879a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:10.757058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9334ca-ba1a-4566-a925-dc686f67879a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:10.765048 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:10.769164 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9334ca-ba1a-4566-a925-dc686f67879a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-db9334ca-ba1a-4566-a925-dc686f67879a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:10.780273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:10.781365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 107/220] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:05:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:10.782185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ff3c4-c953-4252-8a56-38286b09121e None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:10.783071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ff3c4-c953-4252-8a56-38286b09121e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:10.783071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ff3c4-c953-4252-8a56-38286b09121e None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:10.783071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 108/221] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:10.785496 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ff3c4-c953-4252-8a56-38286b09121e None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:10.848795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d3fef-38a7-4d63-9637-c30b4bc91366 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:10.890808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c59b9-0db3-4db9-967f-59a728a69cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:10.891537 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c59b9-0db3-4db9-967f-59a728a69cda None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:10.891826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c59b9-0db3-4db9-967f-59a728a69cda None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:10.892103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c59b9-0db3-4db9-967f-59a728a69cda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:10.909705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ff3c4-c953-4252-8a56-38286b09121e None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:10.952020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbebb92-4da9-4d8b-ac73-2ea6bd63f05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:10.952278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:10.952932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbebb92-4da9-4d8b-ac73-2ea6bd63f05a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:10.953229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbebb92-4da9-4d8b-ac73-2ea6bd63f05a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:10.953773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbebb92-4da9-4d8b-ac73-2ea6bd63f05a None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:10.954456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbebb92-4da9-4d8b-ac73-2ea6bd63f05a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:10.954852 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbebb92-4da9-4d8b-ac73-2ea6bd63f05a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:10.957411 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbebb92-4da9-4d8b-ac73-2ea6bd63f05a None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:10.966643 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 109/222] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:10.967801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888116a2-4568-4ab3-a13f-8e9d5588b152 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:10.968535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888116a2-4568-4ab3-a13f-8e9d5588b152 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:10.968535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888116a2-4568-4ab3-a13f-8e9d5588b152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:10.977609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c59b9-0db3-4db9-967f-59a728a69cda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:10.979464 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c59b9-0db3-4db9-967f-59a728a69cda None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:11.039689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-888116a2-4568-4ab3-a13f-8e9d5588b152 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:11.041977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-888116a2-4568-4ab3-a13f-8e9d5588b152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:11.106744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:11.125129 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 110/223] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:11.126176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888116a2-4568-4ab3-a13f-8e9d5588b152 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:11.165972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 114/224] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:11.171300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c59b9-0db3-4db9-967f-59a728a69cda None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:11.244315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888116a2-4568-4ab3-a13f-8e9d5588b152 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:11.284714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3917f-88bb-46cd-a4fc-ec67edf241d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:11.285341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3917f-88bb-46cd-a4fc-ec67edf241d9 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:11.285623 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3917f-88bb-46cd-a4fc-ec67edf241d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:11.285929 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3917f-88bb-46cd-a4fc-ec67edf241d9 None admin] PATH_INFO: `/v3/users/824da08c89904f77aec01acb63e9cf08` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:11.287956 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3917f-88bb-46cd-a4fc-ec67edf241d9 None admin] RBAC: Authorizing `identity:get_user(user_id=824da08c89904f77aec01acb63e9cf08)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:11.289467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3917f-88bb-46cd-a4fc-ec67edf241d9 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:11.291123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 111/225] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:11 2022] GET /identity/v3/users/824da08c89904f77aec01acb63e9cf08 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:11.296616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3917f-88bb-46cd-a4fc-ec67edf241d9 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:11.366943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:11.390118 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:11.428458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3917f-88bb-46cd-a4fc-ec67edf241d9 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:11.430559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c59b9-0db3-4db9-967f-59a728a69cda None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:11.465007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332e7a1-4315-4142-a7e9-ddca842f841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:11.465733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332e7a1-4315-4142-a7e9-ddca842f841d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:11.465926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332e7a1-4315-4142-a7e9-ddca842f841d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:11.466186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332e7a1-4315-4142-a7e9-ddca842f841d None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:11.467356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332e7a1-4315-4142-a7e9-ddca842f841d None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:11.470159 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332e7a1-4315-4142-a7e9-ddca842f841d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:11.470655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad527a-cb54-4ed1-bd89-277091eea04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:11.472024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad527a-cb54-4ed1-bd89-277091eea04d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:11.472562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 112/226] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:11 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:11.473057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad527a-cb54-4ed1-bd89-277091eea04d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:11.473245 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332e7a1-4315-4142-a7e9-ddca842f841d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:11.473805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad527a-cb54-4ed1-bd89-277091eea04d None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:11.475215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad527a-cb54-4ed1-bd89-277091eea04d None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:11.476923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad527a-cb54-4ed1-bd89-277091eea04d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:11.479194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 115/227] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:11 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:11.480117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad527a-cb54-4ed1-bd89-277091eea04d None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:11.581744 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad527a-cb54-4ed1-bd89-277091eea04d None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:11.583557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332e7a1-4315-4142-a7e9-ddca842f841d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:11.618534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86178f-faad-4c16-af17-dc100da866da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:11.619598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86178f-faad-4c16-af17-dc100da866da None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:11.620046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86178f-faad-4c16-af17-dc100da866da None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:11.620659 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86178f-faad-4c16-af17-dc100da866da None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24/users/824da08c89904f77aec01acb63e9cf08/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:11.622609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86178f-faad-4c16-af17-dc100da866da None admin] RBAC: Authorizing `identity:create_grant(project_id=d120d625e6a84d1c9aa9d1bfab52dd24, user_id=824da08c89904f77aec01acb63e9cf08, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:11.623085 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:11.624523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86178f-faad-4c16-af17-dc100da866da None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:11.632796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4bc6a-fff6-456d-b27d-c144c66e388c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:11.633413 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4bc6a-fff6-456d-b27d-c144c66e388c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:11.633690 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4bc6a-fff6-456d-b27d-c144c66e388c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:11.634719 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4bc6a-fff6-456d-b27d-c144c66e388c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:11.645964 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4bc6a-fff6-456d-b27d-c144c66e388c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-40a4bc6a-fff6-456d-b27d-c144c66e388c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:11.651208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:11.652075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:11.652887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:11.652887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:11.652887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:11.652887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:11.652887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:11.652887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:11.652887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:11.652887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:11.652887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:11.652887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:11.652887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:11.652887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:11.652887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 113/228] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:05:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:11.653571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4bc6a-fff6-456d-b27d-c144c66e388c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:11.760255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4bc6a-fff6-456d-b27d-c144c66e388c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:11.796534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 116/229] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:05:11 2022] PUT /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24/users/824da08c89904f77aec01acb63e9cf08/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:05:11.799864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86178f-faad-4c16-af17-dc100da866da None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:11.803490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25d291-7b73-4da0-898f-57e3696cae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:11.804106 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25d291-7b73-4da0-898f-57e3696cae92 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:11.804489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25d291-7b73-4da0-898f-57e3696cae92 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:11.805535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25d291-7b73-4da0-898f-57e3696cae92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:11.805535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25d291-7b73-4da0-898f-57e3696cae92 None admin] RBAC: Adding query filter params (role.id=4f10622fcf4c4f708cd6b652d886ba3e, scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24, user.id=4ff0650e866b4768a49d211618c7fbd2) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:11.805784 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25d291-7b73-4da0-898f-57e3696cae92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:11.812183 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25d291-7b73-4da0-898f-57e3696cae92 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:11.824741 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 114/230] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:11 2022] GET /identity/v3/role_assignments?user.id=4ff0650e866b4768a49d211618c7fbd2&scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24&role.id=4f10622fcf4c4f708cd6b652d886ba3e => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:11.826566 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:11.967068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86178f-faad-4c16-af17-dc100da866da None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:12.017903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aece3b-223e-43eb-8d36-a21a6b695054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:12.018803 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aece3b-223e-43eb-8d36-a21a6b695054 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:12.019231 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aece3b-223e-43eb-8d36-a21a6b695054 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:12.019649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aece3b-223e-43eb-8d36-a21a6b695054 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:12.020499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aece3b-223e-43eb-8d36-a21a6b695054 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:12.020960 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aece3b-223e-43eb-8d36-a21a6b695054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:12.022516 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aece3b-223e-43eb-8d36-a21a6b695054 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:12.031914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 117/231] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:12.038518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25d291-7b73-4da0-898f-57e3696cae92 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:12.097143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:12.166080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25d291-7b73-4da0-898f-57e3696cae92 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:12.201710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dbdb3-bad5-430b-a092-1e04d8c5f624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:12.202377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dbdb3-bad5-430b-a092-1e04d8c5f624 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:12.202656 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dbdb3-bad5-430b-a092-1e04d8c5f624 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:12.202934 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dbdb3-bad5-430b-a092-1e04d8c5f624 None admin] PATH_INFO: `/v3/users/824da08c89904f77aec01acb63e9cf08` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:12.204197 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dbdb3-bad5-430b-a092-1e04d8c5f624 None admin] RBAC: Authorizing `identity:get_user(user_id=824da08c89904f77aec01acb63e9cf08)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:12.205834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dbdb3-bad5-430b-a092-1e04d8c5f624 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:12.207958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 115/232] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:12 2022] GET /identity/v3/users/824da08c89904f77aec01acb63e9cf08 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:12.213327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aece3b-223e-43eb-8d36-a21a6b695054 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:12.305324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:12.345807 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aece3b-223e-43eb-8d36-a21a6b695054 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:12.386339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a07b3-de5f-468c-8a59-f4b332cc99fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:12.386978 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a07b3-de5f-468c-8a59-f4b332cc99fe None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:12.387257 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a07b3-de5f-468c-8a59-f4b332cc99fe None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:12.387531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a07b3-de5f-468c-8a59-f4b332cc99fe None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:12.388675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a07b3-de5f-468c-8a59-f4b332cc99fe None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:12.390192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a07b3-de5f-468c-8a59-f4b332cc99fe None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:12.392236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 118/233] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:12 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:12.401764 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dbdb3-bad5-430b-a092-1e04d8c5f624 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:12.521693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dbdb3-bad5-430b-a092-1e04d8c5f624 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:12.544656 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:12.572602 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc133e0-cf9a-4ee2-8241-e6856b72ebfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:12.573192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc133e0-cf9a-4ee2-8241-e6856b72ebfd None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:12.573478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc133e0-cf9a-4ee2-8241-e6856b72ebfd None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:12.573753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc133e0-cf9a-4ee2-8241-e6856b72ebfd None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24/users/824da08c89904f77aec01acb63e9cf08/roles/f1d8af7d158e4e02a6e77492178db259` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:12.582527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc133e0-cf9a-4ee2-8241-e6856b72ebfd None admin] RBAC: Authorizing `identity:create_grant(project_id=d120d625e6a84d1c9aa9d1bfab52dd24, user_id=824da08c89904f77aec01acb63e9cf08, role_id=f1d8af7d158e4e02a6e77492178db259)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:12.583983 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc133e0-cf9a-4ee2-8241-e6856b72ebfd None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:12.701755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 116/234] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:05:12 2022] PUT /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24/users/824da08c89904f77aec01acb63e9cf08/roles/f1d8af7d158e4e02a6e77492178db259 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:05:12.983731 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7f47a-93fb-4ea2-9f1c-d7dbdf9e4747 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:12.984290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7f47a-93fb-4ea2-9f1c-d7dbdf9e4747 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:12.984290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7f47a-93fb-4ea2-9f1c-d7dbdf9e4747 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:12.986038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 119/235] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:12.991715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be4a58-d6c3-4586-9549-acae774ec7a4 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:12.992156 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be4a58-d6c3-4586-9549-acae774ec7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:12.992355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be4a58-d6c3-4586-9549-acae774ec7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:13.013719 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-a0be4a58-d6c3-4586-9549-acae774ec7a4 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:13.014924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a0be4a58-d6c3-4586-9549-acae774ec7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:13.092993 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 117/236] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:13.392701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d58412-fa13-43ef-b22f-2d65ee699167 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:13.393040 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d58412-fa13-43ef-b22f-2d65ee699167 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:13.393217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d58412-fa13-43ef-b22f-2d65ee699167 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:13.394795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 120/237] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:13.400043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20ca7b-a170-46e9-b9e7-cb1ce58d34a9 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:13.400363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20ca7b-a170-46e9-b9e7-cb1ce58d34a9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:13.400910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20ca7b-a170-46e9-b9e7-cb1ce58d34a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:13.429195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-9b20ca7b-a170-46e9-b9e7-cb1ce58d34a9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:13.430566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9b20ca7b-a170-46e9-b9e7-cb1ce58d34a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:13.520598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 118/238] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:13.579035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d58412-fa13-43ef-b22f-2d65ee699167 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:13.740852 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d58412-fa13-43ef-b22f-2d65ee699167 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:13.789853 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716020a-2ee7-41c1-b39c-3e4d072c04d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:13.790906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716020a-2ee7-41c1-b39c-3e4d072c04d5 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:13.791425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716020a-2ee7-41c1-b39c-3e4d072c04d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:13.791916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716020a-2ee7-41c1-b39c-3e4d072c04d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:13.793405 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716020a-2ee7-41c1-b39c-3e4d072c04d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:13.795189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716020a-2ee7-41c1-b39c-3e4d072c04d5 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:13.798390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 121/239] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:13.802951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20ca7b-a170-46e9-b9e7-cb1ce58d34a9 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:13.867047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:13.881849 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716020a-2ee7-41c1-b39c-3e4d072c04d5 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:13.897619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:13.965311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20ca7b-a170-46e9-b9e7-cb1ce58d34a9 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:14.031788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abde54-c43a-4fea-8c47-23cf33c4a2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:14.032427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abde54-c43a-4fea-8c47-23cf33c4a2f6 None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:14.032680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abde54-c43a-4fea-8c47-23cf33c4a2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:14.032885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abde54-c43a-4fea-8c47-23cf33c4a2f6 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:14.034129 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abde54-c43a-4fea-8c47-23cf33c4a2f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:14.035592 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abde54-c43a-4fea-8c47-23cf33c4a2f6 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:14.065378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716020a-2ee7-41c1-b39c-3e4d072c04d5 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:14.102385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcdbd9-7975-40fe-a15d-05dc0a0c50b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:14.103025 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcdbd9-7975-40fe-a15d-05dc0a0c50b0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:14.103307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcdbd9-7975-40fe-a15d-05dc0a0c50b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:14.103583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcdbd9-7975-40fe-a15d-05dc0a0c50b0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:14.114360 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bcdbd9-7975-40fe-a15d-05dc0a0c50b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-34bcdbd9-7975-40fe-a15d-05dc0a0c50b0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:14.119790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:14.120716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:14.121497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:14.121497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:14.121497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:14.121497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:14.121497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:14.121497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:14.121497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:14.121497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:14.121497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:14.121497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:14.121497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:14.121497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:14.121497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 122/240] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:05:14.122651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c516e-dcfa-43f5-80b9-c0244c53011c None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:14.122651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c516e-dcfa-43f5-80b9-c0244c53011c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:14.122811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c516e-dcfa-43f5-80b9-c0244c53011c None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:14.124192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 123/241] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:14.128899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c516e-dcfa-43f5-80b9-c0244c53011c None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:14.189334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 119/242] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:05:13 2022] POST /identity/v3/users => generated 293 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:05:14.191025 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ee818-c9af-41a1-a1f0-9b43fea64d7f None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:14.191575 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ee818-c9af-41a1-a1f0-9b43fea64d7f None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:14.192102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ee818-c9af-41a1-a1f0-9b43fea64d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:14.219706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:14.220621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-0e3ee818-c9af-41a1-a1f0-9b43fea64d7f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:14.224505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-0e3ee818-c9af-41a1-a1f0-9b43fea64d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:14.267703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c516e-dcfa-43f5-80b9-c0244c53011c None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:14.317023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 120/243] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:14.324230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6f73e-bcfc-448e-a396-08fa15d1f0d7 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:14.324230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6f73e-bcfc-448e-a396-08fa15d1f0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:14.324230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6f73e-bcfc-448e-a396-08fa15d1f0d7 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:14.324540 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 121/244] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:14.325810 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfcfaf0-4491-4b0e-98b2-62be62f08dd6 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:14.326100 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfcfaf0-4491-4b0e-98b2-62be62f08dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:14.326373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfcfaf0-4491-4b0e-98b2-62be62f08dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:14.331227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a990fd-2520-43b4-8785-bec1b6156791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:14.332240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a990fd-2520-43b4-8785-bec1b6156791 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:14.332719 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a990fd-2520-43b4-8785-bec1b6156791 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:14.333187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a990fd-2520-43b4-8785-bec1b6156791 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:14.334442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a990fd-2520-43b4-8785-bec1b6156791 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:14.336087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a990fd-2520-43b4-8785-bec1b6156791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:14.337906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a990fd-2520-43b4-8785-bec1b6156791 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:14.346602 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ccfcfaf0-4491-4b0e-98b2-62be62f08dd6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:14.347879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ccfcfaf0-4491-4b0e-98b2-62be62f08dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:14.348901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 124/245] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:14.355885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a990fd-2520-43b4-8785-bec1b6156791 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:14.374565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:14.443035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 122/246] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:14.479478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a990fd-2520-43b4-8785-bec1b6156791 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:14.509031 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:14.515313 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9c009-0704-4e9c-82e2-d67b56ab1587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:14.515949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9c009-0704-4e9c-82e2-d67b56ab1587 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:14.516306 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9c009-0704-4e9c-82e2-d67b56ab1587 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:14.516654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9c009-0704-4e9c-82e2-d67b56ab1587 None admin] PATH_INFO: `/v3/users/3105caa64deb489b81d7a208d287359c` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:14.532306 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9c009-0704-4e9c-82e2-d67b56ab1587 None admin] RBAC: Authorizing `identity:get_user(user_id=3105caa64deb489b81d7a208d287359c)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:14.534785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9c009-0704-4e9c-82e2-d67b56ab1587 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:14.536884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 125/247] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:14 2022] GET /identity/v3/users/3105caa64deb489b81d7a208d287359c => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:14.542763 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfcfaf0-4491-4b0e-98b2-62be62f08dd6 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:14.615969 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9c009-0704-4e9c-82e2-d67b56ab1587 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:14.685805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfcfaf0-4491-4b0e-98b2-62be62f08dd6 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:14.740274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6f2d3-1dca-4fc5-9340-005a9f765a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:14.740901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6f2d3-1dca-4fc5-9340-005a9f765a14 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:14.741318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6f2d3-1dca-4fc5-9340-005a9f765a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:14.741729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6f2d3-1dca-4fc5-9340-005a9f765a14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:14.766478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9c009-0704-4e9c-82e2-d67b56ab1587 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:14.779944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:14.813551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58cdf7-2c23-45ed-bb52-c868d538eff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:14.814446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58cdf7-2c23-45ed-bb52-c868d538eff0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:14.814906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58cdf7-2c23-45ed-bb52-c868d538eff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:14.815329 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58cdf7-2c23-45ed-bb52-c868d538eff0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:14.829219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f58cdf7-2c23-45ed-bb52-c868d538eff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-1f58cdf7-2c23-45ed-bb52-c868d538eff0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:14.835408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:14.836571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:14.837618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:14.837618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:14.837618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:14.837618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:14.837618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:14.837618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:14.837618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:14.837618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:14.837618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:14.837618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:14.837618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:14.837618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:14.839873 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 126/248] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:14.840908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58cdf7-2c23-45ed-bb52-c868d538eff0 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:14.846952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6f2d3-1dca-4fc5-9340-005a9f765a14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:14.849105 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6f2d3-1dca-4fc5-9340-005a9f765a14 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:14.902578 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:14.977314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 123/249] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:14.978180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6f2d3-1dca-4fc5-9340-005a9f765a14 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:14.979055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58cdf7-2c23-45ed-bb52-c868d538eff0 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:15.025371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285829d9-53ab-4a38-86f4-a0622cfd29c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:15.026661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285829d9-53ab-4a38-86f4-a0622cfd29c1 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:15.028024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285829d9-53ab-4a38-86f4-a0622cfd29c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:15.028527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285829d9-53ab-4a38-86f4-a0622cfd29c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:15.041549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285829d9-53ab-4a38-86f4-a0622cfd29c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-285829d9-53ab-4a38-86f4-a0622cfd29c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:15.047619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:15.048524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:15.049872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:15.049872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:15.049872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:15.049872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:15.049872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:15.049872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:15.049872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:15.049872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:15.049872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:15.049872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:15.049872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:15.049872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:15.050452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 127/250] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:05:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:15.051332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285829d9-53ab-4a38-86f4-a0622cfd29c1 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:15.089956 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:15.173144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285829d9-53ab-4a38-86f4-a0622cfd29c1 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:15.182689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:15.215902 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba41106-0889-4ad8-ba37-d4cf4b0c6c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:15.216506 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6f2d3-1dca-4fc5-9340-005a9f765a14 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:15.217181 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba41106-0889-4ad8-ba37-d4cf4b0c6c32 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:15.217702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba41106-0889-4ad8-ba37-d4cf4b0c6c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:15.218250 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba41106-0889-4ad8-ba37-d4cf4b0c6c32 None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:15.219717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba41106-0889-4ad8-ba37-d4cf4b0c6c32 None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:15.223269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba41106-0889-4ad8-ba37-d4cf4b0c6c32 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:15.225528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 128/251] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:15 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:15.226680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba41106-0889-4ad8-ba37-d4cf4b0c6c32 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:15.287975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66baf699-691d-4551-8f63-e8c94c1629a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:15.288634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baf699-691d-4551-8f63-e8c94c1629a9 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:15.288917 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baf699-691d-4551-8f63-e8c94c1629a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:15.289192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baf699-691d-4551-8f63-e8c94c1629a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:15.289876 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66baf699-691d-4551-8f63-e8c94c1629a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:15.290170 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66baf699-691d-4551-8f63-e8c94c1629a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:15.291627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66baf699-691d-4551-8f63-e8c94c1629a9 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:15.301547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 124/252] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:15.302675 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66baf699-691d-4551-8f63-e8c94c1629a9 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:15.351403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba41106-0889-4ad8-ba37-d4cf4b0c6c32 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:15.387487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b81dc3-0156-41d5-b576-8d47e77f0b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:15.388106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b81dc3-0156-41d5-b576-8d47e77f0b45 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:15.388414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b81dc3-0156-41d5-b576-8d47e77f0b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:15.388688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b81dc3-0156-41d5-b576-8d47e77f0b45 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:15.389356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b81dc3-0156-41d5-b576-8d47e77f0b45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:15.389660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b81dc3-0156-41d5-b576-8d47e77f0b45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:15.391051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b81dc3-0156-41d5-b576-8d47e77f0b45 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:15.404428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 129/253] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:15.405768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fc020-6f57-403d-9f23-b1bb37abfec2 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:15.406119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fc020-6f57-403d-9f23-b1bb37abfec2 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:15.406439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fc020-6f57-403d-9f23-b1bb37abfec2 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:15.408117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 130/254] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:15.409530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fc020-6f57-403d-9f23-b1bb37abfec2 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:15.422812 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66baf699-691d-4551-8f63-e8c94c1629a9 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:15.446173 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:15.453480 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:15.468615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55b297-1e6b-4df3-a38c-a40ba6496909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:15.469594 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55b297-1e6b-4df3-a38c-a40ba6496909 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:15.470296 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55b297-1e6b-4df3-a38c-a40ba6496909 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:15.470772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55b297-1e6b-4df3-a38c-a40ba6496909 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:15.471663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55b297-1e6b-4df3-a38c-a40ba6496909 None admin] RBAC: Adding query filter params (role.id=7da1608958c94a6dbb98eaf955981d2d, scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24, user.id=3105caa64deb489b81d7a208d287359c) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:15.472196 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55b297-1e6b-4df3-a38c-a40ba6496909 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:15.474486 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55b297-1e6b-4df3-a38c-a40ba6496909 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:15.481739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 125/255] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:15 2022] GET /identity/v3/role_assignments?user.id=3105caa64deb489b81d7a208d287359c&scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24&role.id=7da1608958c94a6dbb98eaf955981d2d => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:15.482429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55b297-1e6b-4df3-a38c-a40ba6496909 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:15.544905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fc020-6f57-403d-9f23-b1bb37abfec2 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:15.656051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55b297-1e6b-4df3-a38c-a40ba6496909 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:15.671656 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cda1a1-59ce-4967-b243-400d1283a16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:15.672695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cda1a1-59ce-4967-b243-400d1283a16f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:15.673209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cda1a1-59ce-4967-b243-400d1283a16f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:15.673774 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cda1a1-59ce-4967-b243-400d1283a16f None admin] PATH_INFO: `/v3/users/4ff0650e866b4768a49d211618c7fbd2` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:15.675699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cda1a1-59ce-4967-b243-400d1283a16f None admin] RBAC: Authorizing `identity:get_user(user_id=4ff0650e866b4768a49d211618c7fbd2)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:15.677733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cda1a1-59ce-4967-b243-400d1283a16f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:15.679652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 131/256] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:15 2022] GET /identity/v3/users/4ff0650e866b4768a49d211618c7fbd2 => generated 297 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:15.690251 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407039ae-a945-425c-a11f-84305d3fc791 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:15.690929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407039ae-a945-425c-a11f-84305d3fc791 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:15.691399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407039ae-a945-425c-a11f-84305d3fc791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:15.703538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8f463-778d-4a05-9c7b-c023640ad570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:15.704769 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8f463-778d-4a05-9c7b-c023640ad570 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:15.705069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8f463-778d-4a05-9c7b-c023640ad570 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:15.710072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8f463-778d-4a05-9c7b-c023640ad570 None admin] PATH_INFO: `/v3/users/824da08c89904f77aec01acb63e9cf08` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:15.715287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8f463-778d-4a05-9c7b-c023640ad570 None admin] RBAC: Authorizing `identity:get_user(user_id=824da08c89904f77aec01acb63e9cf08)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:15.717227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8f463-778d-4a05-9c7b-c023640ad570 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:15.723907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 126/257] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:15 2022] GET /identity/v3/users/824da08c89904f77aec01acb63e9cf08 => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:15.729534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8f463-778d-4a05-9c7b-c023640ad570 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:15.737876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-407039ae-a945-425c-a11f-84305d3fc791 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:15.739140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-407039ae-a945-425c-a11f-84305d3fc791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:15.787229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:15.850497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 132/258] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:15.850686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8f463-778d-4a05-9c7b-c023640ad570 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:15.851124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407039ae-a945-425c-a11f-84305d3fc791 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:15.890433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16c37c-c141-49cc-843f-ae37d04a93a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:15.891131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16c37c-c141-49cc-843f-ae37d04a93a2 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:15.891467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16c37c-c141-49cc-843f-ae37d04a93a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:15.891801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16c37c-c141-49cc-843f-ae37d04a93a2 None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:15.893669 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16c37c-c141-49cc-843f-ae37d04a93a2 None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:15.895362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16c37c-c141-49cc-843f-ae37d04a93a2 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:15.897493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 127/259] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:15 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:15.903752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16c37c-c141-49cc-843f-ae37d04a93a2 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:15.913782 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:16.020461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16c37c-c141-49cc-843f-ae37d04a93a2 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:16.024193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407039ae-a945-425c-a11f-84305d3fc791 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:16.040799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:16.060412 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816740d-f80c-43c8-8c02-776b686550ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:16.061422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816740d-f80c-43c8-8c02-776b686550ca None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:16.061873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816740d-f80c-43c8-8c02-776b686550ca None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:16.062649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816740d-f80c-43c8-8c02-776b686550ca None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24/users/4ff0650e866b4768a49d211618c7fbd2/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:16.065729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816740d-f80c-43c8-8c02-776b686550ca None admin] RBAC: Authorizing `identity:create_grant(project_id=d120d625e6a84d1c9aa9d1bfab52dd24, user_id=4ff0650e866b4768a49d211618c7fbd2, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:16.067507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816740d-f80c-43c8-8c02-776b686550ca None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:16.076951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efd707-e34a-4654-99d5-434e344a67b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:16.078080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efd707-e34a-4654-99d5-434e344a67b9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:16.078508 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efd707-e34a-4654-99d5-434e344a67b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:16.078988 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efd707-e34a-4654-99d5-434e344a67b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:16.167380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efd707-e34a-4654-99d5-434e344a67b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:16.169144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efd707-e34a-4654-99d5-434e344a67b9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:16.216228 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 128/260] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:05:15 2022] PUT /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24/users/4ff0650e866b4768a49d211618c7fbd2/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:05:16.217274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:16.310887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 133/261] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:16.316676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816740d-f80c-43c8-8c02-776b686550ca None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:16.340706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efd707-e34a-4654-99d5-434e344a67b9 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:16.507361 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efd707-e34a-4654-99d5-434e344a67b9 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:16.526164 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816740d-f80c-43c8-8c02-776b686550ca None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:16.581891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdaf54e-3f07-4a05-b313-9e1f617ecc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:16.582173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaf54e-3f07-4a05-b313-9e1f617ecc41 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:16.582453 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaf54e-3f07-4a05-b313-9e1f617ecc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:16.582725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaf54e-3f07-4a05-b313-9e1f617ecc41 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:16.584747 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:16.594637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c3d83-7cc8-4949-b33b-df67b0143ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:16.599423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c3d83-7cc8-4949-b33b-df67b0143ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:16.599423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c3d83-7cc8-4949-b33b-df67b0143ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:16.599423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c3d83-7cc8-4949-b33b-df67b0143ed5 None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:16.599423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c3d83-7cc8-4949-b33b-df67b0143ed5 None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:16.604447 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdaf54e-3f07-4a05-b313-9e1f617ecc41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-0bdaf54e-3f07-4a05-b313-9e1f617ecc41 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:16.606788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:16.607688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:16.608748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:16.608748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:16.608748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:16.608748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:16.608748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:16.608748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:16.608748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:16.608748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:16.608748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:16.608748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:16.608748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:16.608748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:16.608748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 134/262] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:05:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:05:16.615804 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c3d83-7cc8-4949-b33b-df67b0143ed5 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:16.617215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdaf54e-3f07-4a05-b313-9e1f617ecc41 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:16.618527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 129/263] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:16 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:16.624384 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c3d83-7cc8-4949-b33b-df67b0143ed5 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:16.679885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:16.742602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdaf54e-3f07-4a05-b313-9e1f617ecc41 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:16.750708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c3d83-7cc8-4949-b33b-df67b0143ed5 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:16.780448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a8a66-acd0-40e7-baa4-4864e31c5c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:16.781061 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a8a66-acd0-40e7-baa4-4864e31c5c2e None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:16.781342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a8a66-acd0-40e7-baa4-4864e31c5c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:16.781615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a8a66-acd0-40e7-baa4-4864e31c5c2e None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:16.782280 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a8a66-acd0-40e7-baa4-4864e31c5c2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:16.782597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a8a66-acd0-40e7-baa4-4864e31c5c2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:16.784126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a8a66-acd0-40e7-baa4-4864e31c5c2e None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:16.794466 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 135/264] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:16.795902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea3da7-8492-4af7-aa08-c975802f7e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:16.801261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea3da7-8492-4af7-aa08-c975802f7e47 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:16.801261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea3da7-8492-4af7-aa08-c975802f7e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:16.801261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea3da7-8492-4af7-aa08-c975802f7e47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:16.801261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea3da7-8492-4af7-aa08-c975802f7e47 None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24, user.id=824da08c89904f77aec01acb63e9cf08) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:16.801261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea3da7-8492-4af7-aa08-c975802f7e47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:16.801261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea3da7-8492-4af7-aa08-c975802f7e47 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:16.802910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a8a66-acd0-40e7-baa4-4864e31c5c2e None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:16.807997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 130/265] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:16 2022] GET /identity/v3/role_assignments?user.id=824da08c89904f77aec01acb63e9cf08&scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:16.865452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:16.911971 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a8a66-acd0-40e7-baa4-4864e31c5c2e None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:16.948828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476015a-0cdd-4272-8356-e682a08ec82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:16.949472 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476015a-0cdd-4272-8356-e682a08ec82c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:16.949750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476015a-0cdd-4272-8356-e682a08ec82c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:16.950023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476015a-0cdd-4272-8356-e682a08ec82c None admin] PATH_INFO: `/v3/users/824da08c89904f77aec01acb63e9cf08` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:16.951539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476015a-0cdd-4272-8356-e682a08ec82c None admin] RBAC: Authorizing `identity:get_user(user_id=824da08c89904f77aec01acb63e9cf08)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:16.952908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476015a-0cdd-4272-8356-e682a08ec82c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:16.954949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 136/266] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:16 2022] GET /identity/v3/users/824da08c89904f77aec01acb63e9cf08 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:16.961158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea3da7-8492-4af7-aa08-c975802f7e47 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:17.080807 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea3da7-8492-4af7-aa08-c975802f7e47 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:17.112765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:17.138441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43fc9d-2191-40fe-aaeb-43293deef8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:17.139005 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43fc9d-2191-40fe-aaeb-43293deef8eb None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:17.139286 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43fc9d-2191-40fe-aaeb-43293deef8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:17.139609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43fc9d-2191-40fe-aaeb-43293deef8eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:17.142742 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89231c8d-0acd-4d1f-94e3-1733c9eb5b72 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:17.143269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89231c8d-0acd-4d1f-94e3-1733c9eb5b72 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:17.143750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89231c8d-0acd-4d1f-94e3-1733c9eb5b72 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:17.145566 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 137/267] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:17.151966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a2f3a-05aa-4915-8522-85a6bfd25c08 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:17.152569 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a2f3a-05aa-4915-8522-85a6bfd25c08 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:17.153038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a2f3a-05aa-4915-8522-85a6bfd25c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:17.174339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-041a2f3a-05aa-4915-8522-85a6bfd25c08 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:17.175944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-041a2f3a-05aa-4915-8522-85a6bfd25c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:17.215198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43fc9d-2191-40fe-aaeb-43293deef8eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:17.216740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43fc9d-2191-40fe-aaeb-43293deef8eb None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:17.250315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 138/268] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:05:17.354454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 131/269] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:17.359760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a2f3a-05aa-4915-8522-85a6bfd25c08 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:17.443596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a5eba-fbbd-48be-9c44-b3a4e85dabe6 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:17.443944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a5eba-fbbd-48be-9c44-b3a4e85dabe6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:17.444191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a5eba-fbbd-48be-9c44-b3a4e85dabe6 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:17.445957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 132/270] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:17.449931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:17.454826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0becc9a-3f70-4d13-a176-f65791494a0d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:17.455234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0becc9a-3f70-4d13-a176-f65791494a0d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:17.455530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0becc9a-3f70-4d13-a176-f65791494a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:17.493077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e0becc9a-3f70-4d13-a176-f65791494a0d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:17.494294 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e0becc9a-3f70-4d13-a176-f65791494a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:17.496458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:17.544151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a2f3a-05aa-4915-8522-85a6bfd25c08 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:17.601248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c95ea5-9352-45c6-a1fa-1891cbe5c21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:17.602364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c95ea5-9352-45c6-a1fa-1891cbe5c21d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:17.602942 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c95ea5-9352-45c6-a1fa-1891cbe5c21d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:17.603452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c95ea5-9352-45c6-a1fa-1891cbe5c21d None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:17.605192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c95ea5-9352-45c6-a1fa-1891cbe5c21d None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:17.606962 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c95ea5-9352-45c6-a1fa-1891cbe5c21d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:17.608897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 139/271] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:17 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:17.610078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c95ea5-9352-45c6-a1fa-1891cbe5c21d None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:17.625506 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 133/272] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:17.626604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0becc9a-3f70-4d13-a176-f65791494a0d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:17.731966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0becc9a-3f70-4d13-a176-f65791494a0d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:17.771624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c00c9-51d2-4012-a29d-cb2d12a8e7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:17.772493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c00c9-51d2-4012-a29d-cb2d12a8e7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:17.772902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c00c9-51d2-4012-a29d-cb2d12a8e7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:17.773343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c00c9-51d2-4012-a29d-cb2d12a8e7f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:17.774255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c00c9-51d2-4012-a29d-cb2d12a8e7f9 None admin] RBAC: Adding query filter params (role.id=f1d8af7d158e4e02a6e77492178db259, scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24, user.id=824da08c89904f77aec01acb63e9cf08) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:17.774702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c00c9-51d2-4012-a29d-cb2d12a8e7f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:17.776640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c00c9-51d2-4012-a29d-cb2d12a8e7f9 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:17.783603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 134/273] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:17 2022] GET /identity/v3/role_assignments?user.id=824da08c89904f77aec01acb63e9cf08&scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24&role.id=f1d8af7d158e4e02a6e77492178db259 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:17.803494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c95ea5-9352-45c6-a1fa-1891cbe5c21d None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:17.842256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:17.857372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e1398-1aeb-4230-8f74-38e0d79d8648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:17.858024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e1398-1aeb-4230-8f74-38e0d79d8648 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:17.858306 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e1398-1aeb-4230-8f74-38e0d79d8648 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:17.862000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e1398-1aeb-4230-8f74-38e0d79d8648 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:17.862000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e1398-1aeb-4230-8f74-38e0d79d8648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:17.862000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e1398-1aeb-4230-8f74-38e0d79d8648 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:17.863842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 140/274] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:17.870124 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c00c9-51d2-4012-a29d-cb2d12a8e7f9 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:17.873178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:17.985257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c00c9-51d2-4012-a29d-cb2d12a8e7f9 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:18.020773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abff96-c14c-4b95-b359-af795f230376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:18.021381 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abff96-c14c-4b95-b359-af795f230376 None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:18.021655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abff96-c14c-4b95-b359-af795f230376 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:18.021927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abff96-c14c-4b95-b359-af795f230376 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:18.022563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abff96-c14c-4b95-b359-af795f230376 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:18.024021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abff96-c14c-4b95-b359-af795f230376 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:18.060892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e1398-1aeb-4230-8f74-38e0d79d8648 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:18.156041 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 135/275] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:05:17 2022] POST /identity/v3/users => generated 293 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:05:18.182159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e1398-1aeb-4230-8f74-38e0d79d8648 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:18.226291 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0855c-af98-4443-a8c1-57d89170bd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:18.226993 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0855c-af98-4443-a8c1-57d89170bd33 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:18.227274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0855c-af98-4443-a8c1-57d89170bd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:18.227547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0855c-af98-4443-a8c1-57d89170bd33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:18.240055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0855c-af98-4443-a8c1-57d89170bd33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-2af0855c-af98-4443-a8c1-57d89170bd33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:18.245240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:18.246164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:18.246985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:18.246985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:18.246985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:18.246985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:18.246985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:18.246985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:18.246985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:18.246985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:18.246985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:18.246985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:18.246985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:18.246985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:18.247580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5911f5-453f-46a7-8626-1ef6ec6dc47d None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:18.247984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5911f5-453f-46a7-8626-1ef6ec6dc47d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:18.247984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5911f5-453f-46a7-8626-1ef6ec6dc47d None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:18.249157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 141/276] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:05:18.249328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 136/277] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:05:18.255464 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5911f5-453f-46a7-8626-1ef6ec6dc47d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:18.255755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099a9ea-c1c8-4638-85a9-afdbd2a5e844 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:18.256295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099a9ea-c1c8-4638-85a9-afdbd2a5e844 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:18.256295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099a9ea-c1c8-4638-85a9-afdbd2a5e844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:18.280299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f099a9ea-c1c8-4638-85a9-afdbd2a5e844 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:18.282259 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f099a9ea-c1c8-4638-85a9-afdbd2a5e844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:18.323771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:18.336756 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:18.362610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 142/278] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:18.372743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5911f5-453f-46a7-8626-1ef6ec6dc47d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:18.414104 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab1b1a-da86-4242-ad58-2d8d385197b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:18.414722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab1b1a-da86-4242-ad58-2d8d385197b8 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:18.415002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab1b1a-da86-4242-ad58-2d8d385197b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:18.415272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab1b1a-da86-4242-ad58-2d8d385197b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:18.415988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab1b1a-da86-4242-ad58-2d8d385197b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:18.416329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab1b1a-da86-4242-ad58-2d8d385197b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:18.417761 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab1b1a-da86-4242-ad58-2d8d385197b8 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:18.427860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 137/279] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:18.433460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099a9ea-c1c8-4638-85a9-afdbd2a5e844 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:18.547732 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099a9ea-c1c8-4638-85a9-afdbd2a5e844 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:18.584237 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f355b8d-a528-407b-aaa1-6781ea1e57e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:18.584913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f355b8d-a528-407b-aaa1-6781ea1e57e2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:18.585201 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f355b8d-a528-407b-aaa1-6781ea1e57e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:18.585515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f355b8d-a528-407b-aaa1-6781ea1e57e2 None admin] PATH_INFO: `/v3/users/f67f78898ae24697913533a5a4b465d9` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:18.599761 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f355b8d-a528-407b-aaa1-6781ea1e57e2 None admin] RBAC: Authorizing `identity:get_user(user_id=f67f78898ae24697913533a5a4b465d9)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:18.601338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f355b8d-a528-407b-aaa1-6781ea1e57e2 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:18.607990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 143/280] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:18 2022] GET /identity/v3/users/f67f78898ae24697913533a5a4b465d9 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:18.615191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab1b1a-da86-4242-ad58-2d8d385197b8 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:18.642990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f355b8d-a528-407b-aaa1-6781ea1e57e2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:18.701676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:18.748668 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab1b1a-da86-4242-ad58-2d8d385197b8 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:18.770715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:18.772650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f355b8d-a528-407b-aaa1-6781ea1e57e2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:18.796487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5cbf2-60fa-4503-b833-90297cdcb6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:18.797333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5cbf2-60fa-4503-b833-90297cdcb6cb None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:18.797616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5cbf2-60fa-4503-b833-90297cdcb6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:18.797949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5cbf2-60fa-4503-b833-90297cdcb6cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:18.798633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5cbf2-60fa-4503-b833-90297cdcb6cb None admin] RBAC: Adding query filter params (role.id=4f10622fcf4c4f708cd6b652d886ba3e, user.id=f67f78898ae24697913533a5a4b465d9, scope.system=all) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:18.799493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5cbf2-60fa-4503-b833-90297cdcb6cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:18.800985 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5cbf2-60fa-4503-b833-90297cdcb6cb None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:18.814031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 138/281] 10.222.0.85 () {56 vars in 1308 bytes} [Mon Aug 29 20:05:18 2022] GET /identity/v3/role_assignments?user.id=f67f78898ae24697913533a5a4b465d9&scope.system=all&role.id=4f10622fcf4c4f708cd6b652d886ba3e => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:18.833791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85975e-458b-4891-bb09-70f5e0d9f5bd None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:18.834119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85975e-458b-4891-bb09-70f5e0d9f5bd None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:18.834344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85975e-458b-4891-bb09-70f5e0d9f5bd None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:18.835945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 139/282] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:05:18.839028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93557a03-4cdd-41e3-ad36-f48460e95f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:18.839701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93557a03-4cdd-41e3-ad36-f48460e95f75 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:18.839981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93557a03-4cdd-41e3-ad36-f48460e95f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:18.840279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93557a03-4cdd-41e3-ad36-f48460e95f75 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:18.846358 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd01ecc-7b30-48bb-b9aa-1b2dd0341cba None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:18.846650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd01ecc-7b30-48bb-b9aa-1b2dd0341cba None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:18.846897 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd01ecc-7b30-48bb-b9aa-1b2dd0341cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:18.871453 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93557a03-4cdd-41e3-ad36-f48460e95f75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-93557a03-4cdd-41e3-ad36-f48460e95f75 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:18.876671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:18.877649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:18.878427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:18.878427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:18.878427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:18.878427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:18.878427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:18.878427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:18.878427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:18.878427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:18.878427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:18.878427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:18.878427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:18.878427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:18.878427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 144/283] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:18.887165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-2dd01ecc-7b30-48bb-b9aa-1b2dd0341cba None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:18.888443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2dd01ecc-7b30-48bb-b9aa-1b2dd0341cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:18.891738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93557a03-4cdd-41e3-ad36-f48460e95f75 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:19.047108 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:19.052748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93557a03-4cdd-41e3-ad36-f48460e95f75 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:19.053882 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 140/284] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:05:19.107673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7d6b9-0515-40fc-b51e-748d6cf442c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:19.108331 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7d6b9-0515-40fc-b51e-748d6cf442c7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:19.108614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7d6b9-0515-40fc-b51e-748d6cf442c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:19.108892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7d6b9-0515-40fc-b51e-748d6cf442c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:19.109571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7d6b9-0515-40fc-b51e-748d6cf442c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:19.109875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7d6b9-0515-40fc-b51e-748d6cf442c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:19.111827 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7d6b9-0515-40fc-b51e-748d6cf442c7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:19.125459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 145/285] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:19.127025 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd01ecc-7b30-48bb-b9aa-1b2dd0341cba None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:19.271829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd01ecc-7b30-48bb-b9aa-1b2dd0341cba None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:19.288070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:19.308575 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dcbef-0998-4108-afe7-a963f0b23e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:19.309128 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dcbef-0998-4108-afe7-a963f0b23e92 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:19.309403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dcbef-0998-4108-afe7-a963f0b23e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:19.309671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dcbef-0998-4108-afe7-a963f0b23e92 None admin] PATH_INFO: `/v3/users/3105caa64deb489b81d7a208d287359c` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:19.310784 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596dcbef-0998-4108-afe7-a963f0b23e92 None admin] RBAC: Authorizing `identity:get_user(user_id=3105caa64deb489b81d7a208d287359c)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:19.312332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596dcbef-0998-4108-afe7-a963f0b23e92 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:19.314533 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 141/286] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:19 2022] GET /identity/v3/users/3105caa64deb489b81d7a208d287359c => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:19.319973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7d6b9-0515-40fc-b51e-748d6cf442c7 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:19.427736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7d6b9-0515-40fc-b51e-748d6cf442c7 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:19.463618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133619e6-4aad-43b7-81dd-60b6935cd5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:19.464267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133619e6-4aad-43b7-81dd-60b6935cd5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:19.464677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133619e6-4aad-43b7-81dd-60b6935cd5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:19.464954 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133619e6-4aad-43b7-81dd-60b6935cd5d4 None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:19.466089 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133619e6-4aad-43b7-81dd-60b6935cd5d4 None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:19.467631 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133619e6-4aad-43b7-81dd-60b6935cd5d4 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:19.469343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 146/287] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:19 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:19.475420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dcbef-0998-4108-afe7-a963f0b23e92 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:19.530651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133619e6-4aad-43b7-81dd-60b6935cd5d4 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:19.566443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:19.632617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dcbef-0998-4108-afe7-a963f0b23e92 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:19.672145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86269bf-d4f6-4ad9-927e-8c2b71ae8ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:19.673072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86269bf-d4f6-4ad9-927e-8c2b71ae8ab8 None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:19.673607 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86269bf-d4f6-4ad9-927e-8c2b71ae8ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:19.674931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86269bf-d4f6-4ad9-927e-8c2b71ae8ab8 None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24/users/3105caa64deb489b81d7a208d287359c/roles/7da1608958c94a6dbb98eaf955981d2d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:19.678444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86269bf-d4f6-4ad9-927e-8c2b71ae8ab8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d120d625e6a84d1c9aa9d1bfab52dd24, user_id=3105caa64deb489b81d7a208d287359c, role_id=7da1608958c94a6dbb98eaf955981d2d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:19.681037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86269bf-d4f6-4ad9-927e-8c2b71ae8ab8 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:19.705901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133619e6-4aad-43b7-81dd-60b6935cd5d4 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:19.740629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83a040-fbf2-4c42-8753-a376c3869db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:19.741268 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83a040-fbf2-4c42-8753-a376c3869db3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:19.741546 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83a040-fbf2-4c42-8753-a376c3869db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:19.741818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83a040-fbf2-4c42-8753-a376c3869db3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:19.753259 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83a040-fbf2-4c42-8753-a376c3869db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-1f83a040-fbf2-4c42-8753-a376c3869db3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:19.760263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:19.766748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:19.767813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:19.767813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:19.767813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:19.767813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:19.767813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:19.767813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:19.767813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:19.767813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:19.767813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:19.767813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:19.767813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:19.767813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:19.767813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 147/288] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:05:19.768717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 142/289] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:05:19 2022] PUT /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24/users/3105caa64deb489b81d7a208d287359c/roles/7da1608958c94a6dbb98eaf955981d2d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:05:19.768948 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83a040-fbf2-4c42-8753-a376c3869db3 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:19.893277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:19.905543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83a040-fbf2-4c42-8753-a376c3869db3 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:19.964098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c63f38-8fe0-4fe8-b900-18f388e346f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:19.965542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63f38-8fe0-4fe8-b900-18f388e346f3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:19.965830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63f38-8fe0-4fe8-b900-18f388e346f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:19.966120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63f38-8fe0-4fe8-b900-18f388e346f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:19.966794 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c63f38-8fe0-4fe8-b900-18f388e346f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:19.967099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c63f38-8fe0-4fe8-b900-18f388e346f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:19.968617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c63f38-8fe0-4fe8-b900-18f388e346f3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:19.983185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 148/290] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:19.989226 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86269bf-d4f6-4ad9-927e-8c2b71ae8ab8 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:20.036758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:20.098065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86269bf-d4f6-4ad9-927e-8c2b71ae8ab8 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:20.135391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b94c1-0736-4179-b206-c0185d06b848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:20.136034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b94c1-0736-4179-b206-c0185d06b848 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:20.136347 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b94c1-0736-4179-b206-c0185d06b848 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:20.136685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b94c1-0736-4179-b206-c0185d06b848 None admin] PATH_INFO: `/v3/users/4ff0650e866b4768a49d211618c7fbd2` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:20.137828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b94c1-0736-4179-b206-c0185d06b848 None admin] RBAC: Authorizing `identity:get_user(user_id=4ff0650e866b4768a49d211618c7fbd2)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:20.139279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b94c1-0736-4179-b206-c0185d06b848 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:20.141657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 143/291] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:19 2022] GET /identity/v3/users/4ff0650e866b4768a49d211618c7fbd2 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:20.149388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c63f38-8fe0-4fe8-b900-18f388e346f3 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:20.291671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c63f38-8fe0-4fe8-b900-18f388e346f3 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:20.342277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9da71-6fb9-483d-a0b6-1bc1fc066218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:20.342940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9da71-6fb9-483d-a0b6-1bc1fc066218 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:20.343186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9da71-6fb9-483d-a0b6-1bc1fc066218 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:20.343609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9da71-6fb9-483d-a0b6-1bc1fc066218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:20.372417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:20.438398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9da71-6fb9-483d-a0b6-1bc1fc066218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:20.439902 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9da71-6fb9-483d-a0b6-1bc1fc066218 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:20.572335 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 149/292] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:20.577933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b94c1-0736-4179-b206-c0185d06b848 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:20.685136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a1875-420b-4083-9fcf-04179bdad462 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:20.685451 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a1875-420b-4083-9fcf-04179bdad462 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:20.685686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a1875-420b-4083-9fcf-04179bdad462 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:20.687298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 150/293] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:20.693747 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e837df-fa37-4a37-8219-5c6376102520 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:20.694293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e837df-fa37-4a37-8219-5c6376102520 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:20.694766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e837df-fa37-4a37-8219-5c6376102520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:20.722486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-e8e837df-fa37-4a37-8219-5c6376102520 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:20.724909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e8e837df-fa37-4a37-8219-5c6376102520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:20.749908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b94c1-0736-4179-b206-c0185d06b848 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:20.762928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:20.799393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202812d-17e5-4deb-8712-26e3a33becfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:20.800577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202812d-17e5-4deb-8712-26e3a33becfc None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:20.801075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202812d-17e5-4deb-8712-26e3a33becfc None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:20.801541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202812d-17e5-4deb-8712-26e3a33becfc None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:20.804667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0202812d-17e5-4deb-8712-26e3a33becfc None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:20.806389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0202812d-17e5-4deb-8712-26e3a33becfc None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:20.813285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 144/294] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:20 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:20.819709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202812d-17e5-4deb-8712-26e3a33becfc None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:20.827274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:20.858333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 151/295] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:20.860497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4da3a-81dd-4662-af82-d3a78f2a990d None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:20.861007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4da3a-81dd-4662-af82-d3a78f2a990d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:20.862015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4da3a-81dd-4662-af82-d3a78f2a990d None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:20.863919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 152/296] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:20.871030 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ecdfe-7a86-458c-a66e-4473529aaf38 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:20.871030 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ecdfe-7a86-458c-a66e-4473529aaf38 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:20.871030 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ecdfe-7a86-458c-a66e-4473529aaf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:20.888819 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-b18ecdfe-7a86-458c-a66e-4473529aaf38 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:20.890015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b18ecdfe-7a86-458c-a66e-4473529aaf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:20.999303 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202812d-17e5-4deb-8712-26e3a33becfc None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:21.005883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 153/297] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:21.061434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b72518-0f92-45a0-b114-687a24712e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:21.062048 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b72518-0f92-45a0-b114-687a24712e78 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:21.062324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b72518-0f92-45a0-b114-687a24712e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:21.062599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b72518-0f92-45a0-b114-687a24712e78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:21.063328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b72518-0f92-45a0-b114-687a24712e78 None admin] RBAC: Adding query filter params (role.id=4f10622fcf4c4f708cd6b652d886ba3e, scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24, user.id=4ff0650e866b4768a49d211618c7fbd2) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:21.063637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b72518-0f92-45a0-b114-687a24712e78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:21.065642 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b72518-0f92-45a0-b114-687a24712e78 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:21.072031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 145/298] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:20 2022] GET /identity/v3/role_assignments?user.id=4ff0650e866b4768a49d211618c7fbd2&scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24&role.id=4f10622fcf4c4f708cd6b652d886ba3e => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:21.242204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ecdfe-7a86-458c-a66e-4473529aaf38 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:21.268617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b76ab-d8da-47d6-9369-1a8f3dbc7b25 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:21.268790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b76ab-d8da-47d6-9369-1a8f3dbc7b25 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:21.269127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b76ab-d8da-47d6-9369-1a8f3dbc7b25 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:21.270520 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 146/299] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:21.277954 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabecbb-7e73-40ff-8fb3-bf97b421472c None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:21.278241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabecbb-7e73-40ff-8fb3-bf97b421472c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:21.278526 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabecbb-7e73-40ff-8fb3-bf97b421472c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:21.305914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-1cabecbb-7e73-40ff-8fb3-bf97b421472c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:21.307111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-1cabecbb-7e73-40ff-8fb3-bf97b421472c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:21.359454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:21.374014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ecdfe-7a86-458c-a66e-4473529aaf38 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:21.387605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:21.405019 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 147/300] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:21.405760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabecbb-7e73-40ff-8fb3-bf97b421472c None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:21.415058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2122ad5-25d5-44bf-baf9-897705d65c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:21.415918 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2122ad5-25d5-44bf-baf9-897705d65c65 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:21.416464 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2122ad5-25d5-44bf-baf9-897705d65c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:21.416949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2122ad5-25d5-44bf-baf9-897705d65c65 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:21.429074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2122ad5-25d5-44bf-baf9-897705d65c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-d2122ad5-25d5-44bf-baf9-897705d65c65 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:21.434458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:21.435385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:21.436207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:21.436207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:21.436207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:21.436207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:21.436207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:21.436207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:21.436207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:21.436207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:21.436207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:21.436207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:21.436207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:21.436207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:21.438675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 154/301] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:21.445646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2122ad5-25d5-44bf-baf9-897705d65c65 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:21.544820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabecbb-7e73-40ff-8fb3-bf97b421472c None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:21.576041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2122ad5-25d5-44bf-baf9-897705d65c65 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:21.594245 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df7269-feef-40da-98e7-40dfa0a01a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:21.594962 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df7269-feef-40da-98e7-40dfa0a01a65 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:21.595642 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df7269-feef-40da-98e7-40dfa0a01a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:21.595988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df7269-feef-40da-98e7-40dfa0a01a65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:21.597291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df7269-feef-40da-98e7-40dfa0a01a65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:21.598769 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df7269-feef-40da-98e7-40dfa0a01a65 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:21.601028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 148/302] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:21.613225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df7269-feef-40da-98e7-40dfa0a01a65 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:21.623772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa642d-c496-4442-9898-d00c969470e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:21.624689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa642d-c496-4442-9898-d00c969470e6 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:21.625125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa642d-c496-4442-9898-d00c969470e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:21.625521 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa642d-c496-4442-9898-d00c969470e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:21.626297 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa642d-c496-4442-9898-d00c969470e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:21.626756 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa642d-c496-4442-9898-d00c969470e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:21.628764 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa642d-c496-4442-9898-d00c969470e6 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:21.637348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 155/303] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:21.644620 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa642d-c496-4442-9898-d00c969470e6 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:21.662349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:21.688768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:21.732648 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df7269-feef-40da-98e7-40dfa0a01a65 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:21.766532 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa642d-c496-4442-9898-d00c969470e6 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:21.770610 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a106a-f9a2-43f7-b35e-69e804109654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:21.771224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a106a-f9a2-43f7-b35e-69e804109654 None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:21.771500 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a106a-f9a2-43f7-b35e-69e804109654 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:21.771766 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a106a-f9a2-43f7-b35e-69e804109654 None admin] PATH_INFO: `/v3/groups` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:21.772431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a106a-f9a2-43f7-b35e-69e804109654 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:21.774266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a106a-f9a2-43f7-b35e-69e804109654 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:21.802767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6aa93-c468-4263-99f7-fb9e944f432d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:21.803285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6aa93-c468-4263-99f7-fb9e944f432d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:21.803565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6aa93-c468-4263-99f7-fb9e944f432d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:21.803840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6aa93-c468-4263-99f7-fb9e944f432d None admin] PATH_INFO: `/v3/users/b4f9aecf93be4076a7c301a3c9bc509f` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:21.817927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6aa93-c468-4263-99f7-fb9e944f432d None admin] RBAC: Authorizing `identity:get_user(user_id=b4f9aecf93be4076a7c301a3c9bc509f)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:21.819379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6aa93-c468-4263-99f7-fb9e944f432d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:21.821075 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 156/304] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:21 2022] GET /identity/v3/users/b4f9aecf93be4076a7c301a3c9bc509f => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:21.821824 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6aa93-c468-4263-99f7-fb9e944f432d None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:21.880198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 149/305] 10.222.0.85 () {60 vars in 1134 bytes} [Mon Aug 29 20:05:21 2022] POST /identity/v3/groups => generated 229 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:05:21.880198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a106a-f9a2-43f7-b35e-69e804109654 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:21.938179 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:21.949159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6aa93-c468-4263-99f7-fb9e944f432d None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:21.994347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afaff8-c771-469b-858a-9947385be4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:21.995346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afaff8-c771-469b-858a-9947385be4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:21.995761 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afaff8-c771-469b-858a-9947385be4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:21.996245 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afaff8-c771-469b-858a-9947385be4a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:21.999256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:22.003005 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a106a-f9a2-43f7-b35e-69e804109654 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:22.011009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74afaff8-c771-469b-858a-9947385be4a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-74afaff8-c771-469b-858a-9947385be4a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:22.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:22.017429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:22.018193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:22.018193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:22.018193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:22.018193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:22.018193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:22.018193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:22.018193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:22.018193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:22.018193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:22.018193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:22.018193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:22.018193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:22.021086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 157/306] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:22.026715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afaff8-c771-469b-858a-9947385be4a2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:22.055176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0b60d-adf1-48f9-bcb6-fc1708ceee67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:22.055176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0b60d-adf1-48f9-bcb6-fc1708ceee67 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:22.055176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0b60d-adf1-48f9-bcb6-fc1708ceee67 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:22.055176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0b60d-adf1-48f9-bcb6-fc1708ceee67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:22.055543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0b60d-adf1-48f9-bcb6-fc1708ceee67 None admin] RBAC: Adding query filter params (role.id=7da1608958c94a6dbb98eaf955981d2d, user.id=b4f9aecf93be4076a7c301a3c9bc509f, scope.system=all) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:22.055543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0b60d-adf1-48f9-bcb6-fc1708ceee67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:22.058586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0b60d-adf1-48f9-bcb6-fc1708ceee67 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:22.066327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 150/307] 10.222.0.85 () {56 vars in 1308 bytes} [Mon Aug 29 20:05:21 2022] GET /identity/v3/role_assignments?user.id=b4f9aecf93be4076a7c301a3c9bc509f&scope.system=all&role.id=7da1608958c94a6dbb98eaf955981d2d => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:22.159002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afaff8-c771-469b-858a-9947385be4a2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:22.201529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d2a8b-90b3-4806-ba7f-c744157ecf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:22.202205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d2a8b-90b3-4806-ba7f-c744157ecf16 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:22.202558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d2a8b-90b3-4806-ba7f-c744157ecf16 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:22.202939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d2a8b-90b3-4806-ba7f-c744157ecf16 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:22.203632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d2a8b-90b3-4806-ba7f-c744157ecf16 None admin] RBAC: Adding query filter params (name=member) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:22.203937 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d2a8b-90b3-4806-ba7f-c744157ecf16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:22.206213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d2a8b-90b3-4806-ba7f-c744157ecf16 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:22.216641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 158/308] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:22.222807 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0b60d-adf1-48f9-bcb6-fc1708ceee67 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:22.317479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721deb10-a1cb-4fc9-8822-f7ad91546a37 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:22.318194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721deb10-a1cb-4fc9-8822-f7ad91546a37 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:22.319028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721deb10-a1cb-4fc9-8822-f7ad91546a37 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:22.321382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 159/309] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:22.326830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d247d-4ed5-45ae-9f98-9e2b373778c9 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:22.327118 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d247d-4ed5-45ae-9f98-9e2b373778c9 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:22.327508 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d247d-4ed5-45ae-9f98-9e2b373778c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:22.331617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0b60d-adf1-48f9-bcb6-fc1708ceee67 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:22.350991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-b86d247d-4ed5-45ae-9f98-9e2b373778c9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:22.352478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b86d247d-4ed5-45ae-9f98-9e2b373778c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:22.360881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:22.371875 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1c854-a8d2-4136-bed1-2bb1a315fdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:22.372536 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1c854-a8d2-4136-bed1-2bb1a315fdbd None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:22.372867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1c854-a8d2-4136-bed1-2bb1a315fdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:22.373144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1c854-a8d2-4136-bed1-2bb1a315fdbd None admin] PATH_INFO: `/v3/users/f67f78898ae24697913533a5a4b465d9` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:22.374862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1c854-a8d2-4136-bed1-2bb1a315fdbd None admin] RBAC: Authorizing `identity:get_user(user_id=f67f78898ae24697913533a5a4b465d9)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:22.378097 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1c854-a8d2-4136-bed1-2bb1a315fdbd None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:22.378923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 151/310] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:22 2022] GET /identity/v3/users/f67f78898ae24697913533a5a4b465d9 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:22.384966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1c854-a8d2-4136-bed1-2bb1a315fdbd None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:22.426320 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:22.445792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 160/311] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:22.504632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1c854-a8d2-4136-bed1-2bb1a315fdbd None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:22.540590 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810c750-fac0-4de2-a081-b2c5b50746fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:22.541251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810c750-fac0-4de2-a081-b2c5b50746fd None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:22.541535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810c750-fac0-4de2-a081-b2c5b50746fd None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:22.541808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810c750-fac0-4de2-a081-b2c5b50746fd None admin] PATH_INFO: `/v3/system/users/f67f78898ae24697913533a5a4b465d9/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:22.543267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810c750-fac0-4de2-a081-b2c5b50746fd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f67f78898ae24697913533a5a4b465d9, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:22.544717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810c750-fac0-4de2-a081-b2c5b50746fd None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:22.673468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 152/312] 10.222.0.85 () {58 vars in 1334 bytes} [Mon Aug 29 20:05:22 2022] PUT /identity/v3/system/users/f67f78898ae24697913533a5a4b465d9/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:05:22.713062 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d247d-4ed5-45ae-9f98-9e2b373778c9 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:22.827696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d247d-4ed5-45ae-9f98-9e2b373778c9 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:22.864316 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf6280-796b-4851-8c5a-0c542fe4fa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:22.864870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf6280-796b-4851-8c5a-0c542fe4fa56 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:22.865153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf6280-796b-4851-8c5a-0c542fe4fa56 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:22.865430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf6280-796b-4851-8c5a-0c542fe4fa56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:22.876464 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf6280-796b-4851-8c5a-0c542fe4fa56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-f3bf6280-796b-4851-8c5a-0c542fe4fa56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:22.882018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:22.882872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:22.883646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:22.883646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:22.883646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:22.883646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:22.883646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:22.883646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:22.883646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:22.883646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:22.883646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:22.883646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:22.883646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:22.883646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:22.883646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 161/313] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:22.890275 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810c750-fac0-4de2-a081-b2c5b50746fd None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:22.998701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:23.007035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810c750-fac0-4de2-a081-b2c5b50746fd None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:23.043135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bd1d8-054e-4fa0-9579-013064f6f1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:23.043992 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bd1d8-054e-4fa0-9579-013064f6f1bc None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:23.044441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bd1d8-054e-4fa0-9579-013064f6f1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:23.044852 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bd1d8-054e-4fa0-9579-013064f6f1bc None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:23.045699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bd1d8-054e-4fa0-9579-013064f6f1bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:23.046141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bd1d8-054e-4fa0-9579-013064f6f1bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:23.047746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bd1d8-054e-4fa0-9579-013064f6f1bc None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:23.063404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 153/314] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:23.072252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf6280-796b-4851-8c5a-0c542fe4fa56 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:23.100923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:23.179379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf6280-796b-4851-8c5a-0c542fe4fa56 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:23.215560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d91c75-be1a-4cd6-af68-728f29202cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:23.216288 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d91c75-be1a-4cd6-af68-728f29202cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:23.216572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d91c75-be1a-4cd6-af68-728f29202cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:23.216941 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d91c75-be1a-4cd6-af68-728f29202cc6 None admin] PATH_INFO: `/v3/users/3105caa64deb489b81d7a208d287359c` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:23.218106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d91c75-be1a-4cd6-af68-728f29202cc6 None admin] RBAC: Authorizing `identity:get_user(user_id=3105caa64deb489b81d7a208d287359c)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:23.219550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d91c75-be1a-4cd6-af68-728f29202cc6 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:23.222082 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 162/315] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:23 2022] GET /identity/v3/users/3105caa64deb489b81d7a208d287359c => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:23.227815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bd1d8-054e-4fa0-9579-013064f6f1bc None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:23.349680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bd1d8-054e-4fa0-9579-013064f6f1bc None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:23.385108 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4b1dd-de66-4625-815a-3ed1c301d147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:23.386002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4b1dd-de66-4625-815a-3ed1c301d147 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:23.386454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4b1dd-de66-4625-815a-3ed1c301d147 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:23.386897 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4b1dd-de66-4625-815a-3ed1c301d147 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:23.431311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:23.455987 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4b1dd-de66-4625-815a-3ed1c301d147 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:23.457461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4b1dd-de66-4625-815a-3ed1c301d147 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:23.556561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 154/316] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:23.562201 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d91c75-be1a-4cd6-af68-728f29202cc6 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:23.640299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:23.676109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d91c75-be1a-4cd6-af68-728f29202cc6 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:23.717873 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80ded8-bff3-4f8a-b3f3-6b91db65e40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:23.718832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80ded8-bff3-4f8a-b3f3-6b91db65e40a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:23.719306 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80ded8-bff3-4f8a-b3f3-6b91db65e40a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:23.719790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80ded8-bff3-4f8a-b3f3-6b91db65e40a None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:23.721239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80ded8-bff3-4f8a-b3f3-6b91db65e40a None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:23.722979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80ded8-bff3-4f8a-b3f3-6b91db65e40a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:23.725189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 163/317] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:23 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:23.731331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4b1dd-de66-4625-815a-3ed1c301d147 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:23.793320 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:23.838811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4b1dd-de66-4625-815a-3ed1c301d147 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:23.874772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae395811-347c-47b1-b692-8a5fa287cbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:23.875392 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae395811-347c-47b1-b692-8a5fa287cbdc None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:23.875657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae395811-347c-47b1-b692-8a5fa287cbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:23.875928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae395811-347c-47b1-b692-8a5fa287cbdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:23.876716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae395811-347c-47b1-b692-8a5fa287cbdc None admin] RBAC: Adding query filter params (role.id=7da1608958c94a6dbb98eaf955981d2d, scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24, user.id=3105caa64deb489b81d7a208d287359c) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:23.877024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae395811-347c-47b1-b692-8a5fa287cbdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:23.878473 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae395811-347c-47b1-b692-8a5fa287cbdc None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:23.884961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 155/318] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:23 2022] GET /identity/v3/role_assignments?user.id=3105caa64deb489b81d7a208d287359c&scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24&role.id=7da1608958c94a6dbb98eaf955981d2d => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:24.025897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6201b3d-d245-433e-b7f4-b6829ffdb24e None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:24.026509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6201b3d-d245-433e-b7f4-b6829ffdb24e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:24.026979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6201b3d-d245-433e-b7f4-b6829ffdb24e None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:24.028863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 164/319] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:24.034412 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd49bb6-28cd-41ae-8a54-65ef7b6e6cb4 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:24.034697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd49bb6-28cd-41ae-8a54-65ef7b6e6cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:24.034966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd49bb6-28cd-41ae-8a54-65ef7b6e6cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:24.054136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-afd49bb6-28cd-41ae-8a54-65ef7b6e6cb4 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:24.055325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-afd49bb6-28cd-41ae-8a54-65ef7b6e6cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:24.125068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9309ae-02e4-4725-a46e-c23925032bbc None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:24.125422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9309ae-02e4-4725-a46e-c23925032bbc None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:24.125623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9309ae-02e4-4725-a46e-c23925032bbc None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:24.128675 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 156/321] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:24.128792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 165/321] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:24.133609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab054e9c-7056-4c7d-b054-d4c2995791c7 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:24.133955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab054e9c-7056-4c7d-b054-d4c2995791c7 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:24.134232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab054e9c-7056-4c7d-b054-d4c2995791c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:24.153935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ab054e9c-7056-4c7d-b054-d4c2995791c7 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:24.155415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ab054e9c-7056-4c7d-b054-d4c2995791c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:24.205202 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:24.225950 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 157/322] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:24.400859 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9309ae-02e4-4725-a46e-c23925032bbc None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:24.502539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9309ae-02e4-4725-a46e-c23925032bbc None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:24.505253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab054e9c-7056-4c7d-b054-d4c2995791c7 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:24.542851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc8730-e8b3-4eee-bd2c-a435edc67837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:24.543745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc8730-e8b3-4eee-bd2c-a435edc67837 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:24.544033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc8730-e8b3-4eee-bd2c-a435edc67837 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:24.544344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc8730-e8b3-4eee-bd2c-a435edc67837 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:24.545516 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc8730-e8b3-4eee-bd2c-a435edc67837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:24.547022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc8730-e8b3-4eee-bd2c-a435edc67837 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:24.552356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 166/323] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:24.556993 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc8730-e8b3-4eee-bd2c-a435edc67837 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:24.569428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:24.631181 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab054e9c-7056-4c7d-b054-d4c2995791c7 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:24.667351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce81d6-92e9-4160-a658-cbc3661d545f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:24.667984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce81d6-92e9-4160-a658-cbc3661d545f None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:24.668293 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce81d6-92e9-4160-a658-cbc3661d545f None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:24.668557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce81d6-92e9-4160-a658-cbc3661d545f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:24.679183 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce81d6-92e9-4160-a658-cbc3661d545f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-d7ce81d6-92e9-4160-a658-cbc3661d545f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:24.684096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:24.685143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:24.686134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:24.686134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:24.686134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:24.686134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:24.686134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:24.686134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:24.686134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:24.686134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:24.686134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:24.686134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:24.686134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:24.686134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:24.687050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc8730-e8b3-4eee-bd2c-a435edc67837 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:24.687137 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 158/324] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:05:24.687216 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba7e38-a2fb-4a36-b704-cb6856b1c6c5 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:24.687652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba7e38-a2fb-4a36-b704-cb6856b1c6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:24.687771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba7e38-a2fb-4a36-b704-cb6856b1c6c5 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:24.689726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 159/325] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:24.691658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba7e38-a2fb-4a36-b704-cb6856b1c6c5 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:24.724010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977696d8-b07b-4588-a864-86be83201905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:24.724674 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977696d8-b07b-4588-a864-86be83201905 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:24.724955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977696d8-b07b-4588-a864-86be83201905 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:24.725224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977696d8-b07b-4588-a864-86be83201905 None admin] PATH_INFO: `/v3/groups` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:24.725855 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977696d8-b07b-4588-a864-86be83201905 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:24.727286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977696d8-b07b-4588-a864-86be83201905 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:24.739668 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:24.795639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba7e38-a2fb-4a36-b704-cb6856b1c6c5 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:24.828628 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf1106-a1fa-4097-85b0-f48c09e877fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:24.829228 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf1106-a1fa-4097-85b0-f48c09e877fd None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:24.829504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf1106-a1fa-4097-85b0-f48c09e877fd None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:24.829775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf1106-a1fa-4097-85b0-f48c09e877fd None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:24.831039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf1106-a1fa-4097-85b0-f48c09e877fd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:24.831336 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf1106-a1fa-4097-85b0-f48c09e877fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:24.832796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf1106-a1fa-4097-85b0-f48c09e877fd None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:24.841076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 160/326] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:24.842200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975c295-6443-4b21-8694-c4b133b0ef5a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:24.842477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975c295-6443-4b21-8694-c4b133b0ef5a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:24.842744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975c295-6443-4b21-8694-c4b133b0ef5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:24.860185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d975c295-6443-4b21-8694-c4b133b0ef5a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:24.860707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 167/327] 10.222.0.85 () {60 vars in 1134 bytes} [Mon Aug 29 20:05:24 2022] POST /identity/v3/groups => generated 226 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:05:24.861428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977696d8-b07b-4588-a864-86be83201905 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:24.861528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d975c295-6443-4b21-8694-c4b133b0ef5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:24.941488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 161/328] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:24.948797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:24.986367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977696d8-b07b-4588-a864-86be83201905 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:25.025780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8940a9-b65f-4c57-ae17-713f5210b687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:25.026755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8940a9-b65f-4c57-ae17-713f5210b687 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:25.027035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8940a9-b65f-4c57-ae17-713f5210b687 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:25.027309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8940a9-b65f-4c57-ae17-713f5210b687 None admin] PATH_INFO: `/v3/users/b4f9aecf93be4076a7c301a3c9bc509f` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:25.028595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8940a9-b65f-4c57-ae17-713f5210b687 None admin] RBAC: Authorizing `identity:get_user(user_id=b4f9aecf93be4076a7c301a3c9bc509f)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:25.030053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8940a9-b65f-4c57-ae17-713f5210b687 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:25.031751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 168/329] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:24 2022] GET /identity/v3/users/b4f9aecf93be4076a7c301a3c9bc509f => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:25.037680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975c295-6443-4b21-8694-c4b133b0ef5a None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:25.075461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:25.154203 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975c295-6443-4b21-8694-c4b133b0ef5a None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:25.189818 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eebb68-4569-473e-aa7a-526f1dfc8b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:25.190645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eebb68-4569-473e-aa7a-526f1dfc8b1d None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:25.190928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eebb68-4569-473e-aa7a-526f1dfc8b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:25.191200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eebb68-4569-473e-aa7a-526f1dfc8b1d None admin] PATH_INFO: `/v3/system/users/b4f9aecf93be4076a7c301a3c9bc509f/roles/7da1608958c94a6dbb98eaf955981d2d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:25.192694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eebb68-4569-473e-aa7a-526f1dfc8b1d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b4f9aecf93be4076a7c301a3c9bc509f, role_id=7da1608958c94a6dbb98eaf955981d2d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:25.194012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eebb68-4569-473e-aa7a-526f1dfc8b1d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:25.216788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8940a9-b65f-4c57-ae17-713f5210b687 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:25.268203 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 162/330] 10.222.0.85 () {58 vars in 1334 bytes} [Mon Aug 29 20:05:25 2022] PUT /identity/v3/system/users/b4f9aecf93be4076a7c301a3c9bc509f/roles/7da1608958c94a6dbb98eaf955981d2d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:05:25.346854 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8940a9-b65f-4c57-ae17-713f5210b687 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:25.386329 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6edfd-6d32-47a5-adf9-cfcddfab1e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:25.386986 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6edfd-6d32-47a5-adf9-cfcddfab1e31 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:25.387265 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6edfd-6d32-47a5-adf9-cfcddfab1e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:25.387574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6edfd-6d32-47a5-adf9-cfcddfab1e31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:25.389757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:25.401227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6edfd-6d32-47a5-adf9-cfcddfab1e31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-4fd6edfd-6d32-47a5-adf9-cfcddfab1e31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:25.406662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:25.407602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:25.409332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:25.409332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:25.409332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:25.409332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:25.409332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:25.409332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:25.409332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:25.409332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:25.409332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:25.409332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:25.409332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:25.409332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:25.409332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 169/331] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:05:25.414681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eebb68-4569-473e-aa7a-526f1dfc8b1d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:25.536713 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:25.540192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eebb68-4569-473e-aa7a-526f1dfc8b1d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:25.579392 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef350155-429d-46e8-8229-2834b353ca4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:25.580764 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef350155-429d-46e8-8229-2834b353ca4d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:25.581052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef350155-429d-46e8-8229-2834b353ca4d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:25.581391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef350155-429d-46e8-8229-2834b353ca4d None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:25.582385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef350155-429d-46e8-8229-2834b353ca4d None admin] RBAC: Adding query filter params (name=member) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:25.582487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef350155-429d-46e8-8229-2834b353ca4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:25.583912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef350155-429d-46e8-8229-2834b353ca4d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:25.593556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 163/332] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:25.598990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6edfd-6d32-47a5-adf9-cfcddfab1e31 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:25.708442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6edfd-6d32-47a5-adf9-cfcddfab1e31 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:25.744577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d9cb5-a81a-4181-8826-1cd216f2c3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:25.745231 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d9cb5-a81a-4181-8826-1cd216f2c3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:25.745518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d9cb5-a81a-4181-8826-1cd216f2c3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:25.745773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d9cb5-a81a-4181-8826-1cd216f2c3d8 None admin] PATH_INFO: `/v3/users/f67f78898ae24697913533a5a4b465d9` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:25.746935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d9cb5-a81a-4181-8826-1cd216f2c3d8 None admin] RBAC: Authorizing `identity:get_user(user_id=f67f78898ae24697913533a5a4b465d9)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:25.748419 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d9cb5-a81a-4181-8826-1cd216f2c3d8 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:25.750359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 170/333] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:25 2022] GET /identity/v3/users/f67f78898ae24697913533a5a4b465d9 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:25.756004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef350155-429d-46e8-8229-2834b353ca4d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:25.864676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef350155-429d-46e8-8229-2834b353ca4d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:25.901054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d3e69-6a50-4a79-b257-ad985df62240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:25.901667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d3e69-6a50-4a79-b257-ad985df62240 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:25.901950 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d3e69-6a50-4a79-b257-ad985df62240 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:25.902227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d3e69-6a50-4a79-b257-ad985df62240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:25.903009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d3e69-6a50-4a79-b257-ad985df62240 None admin] RBAC: Adding query filter params (role.id=4f10622fcf4c4f708cd6b652d886ba3e, user.id=f67f78898ae24697913533a5a4b465d9, scope.system=all) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:25.903321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d3e69-6a50-4a79-b257-ad985df62240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:25.904924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d3e69-6a50-4a79-b257-ad985df62240 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:25.918433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 164/334] 10.222.0.85 () {56 vars in 1308 bytes} [Mon Aug 29 20:05:25 2022] GET /identity/v3/role_assignments?user.id=f67f78898ae24697913533a5a4b465d9&scope.system=all&role.id=4f10622fcf4c4f708cd6b652d886ba3e => generated 510 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:27.269580 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4495c-2459-419f-a103-8133e017ee02 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:27.270101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4495c-2459-419f-a103-8133e017ee02 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:27.270101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4495c-2459-419f-a103-8133e017ee02 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:27.271622 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 171/335] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:27.277142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d69300-134c-44d1-9c94-1c8e43237139 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:27.277614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d69300-134c-44d1-9c94-1c8e43237139 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:27.277813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d69300-134c-44d1-9c94-1c8e43237139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:27.296826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d2d69300-134c-44d1-9c94-1c8e43237139 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:27.298095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d2d69300-134c-44d1-9c94-1c8e43237139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:27.315225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:27.382562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 165/336] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:27.395191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db6493-864d-4100-b440-9759affd69e4 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:27.395500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db6493-864d-4100-b440-9759affd69e4 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:27.395780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db6493-864d-4100-b440-9759affd69e4 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:27.397397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 172/337] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:27.403272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d01cc-9cdd-4d24-a1aa-cd3cdc7543a1 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:27.403557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d01cc-9cdd-4d24-a1aa-cd3cdc7543a1 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:27.403851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d01cc-9cdd-4d24-a1aa-cd3cdc7543a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:27.428092 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-bd6d01cc-9cdd-4d24-a1aa-cd3cdc7543a1 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:27.429335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bd6d01cc-9cdd-4d24-a1aa-cd3cdc7543a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:27.503434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 166/338] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:27.841445 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db6493-864d-4100-b440-9759affd69e4 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:27.881828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d01cc-9cdd-4d24-a1aa-cd3cdc7543a1 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:27.918341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:27.977946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db6493-864d-4100-b440-9759affd69e4 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:28.116418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47b2fd-4d09-4142-84f7-185e142c237f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:28.117213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47b2fd-4d09-4142-84f7-185e142c237f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:28.117496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47b2fd-4d09-4142-84f7-185e142c237f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:28.117766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47b2fd-4d09-4142-84f7-185e142c237f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:28.134303 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47b2fd-4d09-4142-84f7-185e142c237f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-fa47b2fd-4d09-4142-84f7-185e142c237f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:28.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:28.143385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:28.144236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:28.144236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:28.144236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:28.144236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:28.144236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:28.144236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:28.144236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:28.144236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:28.144236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:28.144236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:28.144236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:28.144236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:28.144844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:28.144914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 173/339] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:28.144914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc1f0c-21e7-4515-93e3-433d87367871 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:28.145269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc1f0c-21e7-4515-93e3-433d87367871 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:28.145583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc1f0c-21e7-4515-93e3-433d87367871 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:28.146835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 174/340] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:28.151626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc1f0c-21e7-4515-93e3-433d87367871 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:28.180947 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d01cc-9cdd-4d24-a1aa-cd3cdc7543a1 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:28.222827 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8c916-859d-4a67-9145-74463e0d2dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:28.223529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8c916-859d-4a67-9145-74463e0d2dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:28.223854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8c916-859d-4a67-9145-74463e0d2dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:28.224127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8c916-859d-4a67-9145-74463e0d2dc2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:28.239517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8c916-859d-4a67-9145-74463e0d2dc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-d9c8c916-859d-4a67-9145-74463e0d2dc2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:28.246215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:28.250208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 167/341] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:05:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:05:28.251021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf68f27-f763-429c-b0fd-d4ec51712e58 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:28.253002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf68f27-f763-429c-b0fd-d4ec51712e58 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:28.253002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf68f27-f763-429c-b0fd-d4ec51712e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:28.272550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-7bf68f27-f763-429c-b0fd-d4ec51712e58 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:28.273757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7bf68f27-f763-429c-b0fd-d4ec51712e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:28.299144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc1f0c-21e7-4515-93e3-433d87367871 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:28.346416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:28.374336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcdfc8-87dc-4222-b2bf-063e500e1b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:28.375159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdfc8-87dc-4222-b2bf-063e500e1b62 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:28.375442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdfc8-87dc-4222-b2bf-063e500e1b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:28.375737 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdfc8-87dc-4222-b2bf-063e500e1b62 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:28.377133 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcdfc8-87dc-4222-b2bf-063e500e1b62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:28.377438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcdfc8-87dc-4222-b2bf-063e500e1b62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:28.384375 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcdfc8-87dc-4222-b2bf-063e500e1b62 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:28.385089 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 168/342] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:28.386717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf68f27-f763-429c-b0fd-d4ec51712e58 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:28.394340 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 175/343] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:28.404552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcdfc8-87dc-4222-b2bf-063e500e1b62 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:28.469495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:28.505873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf68f27-f763-429c-b0fd-d4ec51712e58 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:28.544219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf5b97-d365-4d37-90d7-8316b6bde6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:28.544859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf5b97-d365-4d37-90d7-8316b6bde6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:28.545181 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf5b97-d365-4d37-90d7-8316b6bde6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:28.545450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf5b97-d365-4d37-90d7-8316b6bde6c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:28.546474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf5b97-d365-4d37-90d7-8316b6bde6c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:28.546893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf5b97-d365-4d37-90d7-8316b6bde6c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:28.548324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf5b97-d365-4d37-90d7-8316b6bde6c1 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:28.557651 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 169/344] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:28.559065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbd8b7-f058-4ee8-afb9-8901efff8e31 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:28.559350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbd8b7-f058-4ee8-afb9-8901efff8e31 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:28.559620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbd8b7-f058-4ee8-afb9-8901efff8e31 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:28.560871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 170/345] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:28.567269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aec2bf-c866-45a4-93b9-e9d57a1df9fd None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:28.567518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aec2bf-c866-45a4-93b9-e9d57a1df9fd None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:28.567773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aec2bf-c866-45a4-93b9-e9d57a1df9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:28.587758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-a1aec2bf-c866-45a4-93b9-e9d57a1df9fd None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:28.588968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a1aec2bf-c866-45a4-93b9-e9d57a1df9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:28.625201 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcdfc8-87dc-4222-b2bf-063e500e1b62 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:28.666732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 171/346] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:05:28.667702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aec2bf-c866-45a4-93b9-e9d57a1df9fd None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:28.685759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb18a1c-f0d0-48be-973a-22c11a24eee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:28.686683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb18a1c-f0d0-48be-973a-22c11a24eee7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:28.687146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb18a1c-f0d0-48be-973a-22c11a24eee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:28.687628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb18a1c-f0d0-48be-973a-22c11a24eee7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:28.774185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:28.827017 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:28.831787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aec2bf-c866-45a4-93b9-e9d57a1df9fd None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:28.849915 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb18a1c-f0d0-48be-973a-22c11a24eee7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:28.851397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb18a1c-f0d0-48be-973a-22c11a24eee7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:28.889874 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6226e8-cdcf-489f-b4a5-745ee5cde103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:28.890572 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6226e8-cdcf-489f-b4a5-745ee5cde103 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:28.890853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6226e8-cdcf-489f-b4a5-745ee5cde103 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:28.891119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6226e8-cdcf-489f-b4a5-745ee5cde103 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:29.010053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 176/347] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:29.015962 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6226e8-cdcf-489f-b4a5-745ee5cde103 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:29.016235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb18a1c-f0d0-48be-973a-22c11a24eee7 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:29.018042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6226e8-cdcf-489f-b4a5-745ee5cde103 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:29.124045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:29.130627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:29.147809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb18a1c-f0d0-48be-973a-22c11a24eee7 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:29.179391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 172/348] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:29.180420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6226e8-cdcf-489f-b4a5-745ee5cde103 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:29.190748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc74aae-55c2-40dd-8035-3608de78f952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:29.191821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc74aae-55c2-40dd-8035-3608de78f952 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:29.192510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc74aae-55c2-40dd-8035-3608de78f952 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:29.193076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc74aae-55c2-40dd-8035-3608de78f952 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:29.194530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc74aae-55c2-40dd-8035-3608de78f952 None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:29.195987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc74aae-55c2-40dd-8035-3608de78f952 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:29.198016 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 177/349] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:29 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:29.198800 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc74aae-55c2-40dd-8035-3608de78f952 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:29.303295 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6226e8-cdcf-489f-b4a5-745ee5cde103 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:29.347247 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa3c19-b7b1-4d0f-ad56-e14220bd6819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:29.347920 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa3c19-b7b1-4d0f-ad56-e14220bd6819 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:29.348217 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa3c19-b7b1-4d0f-ad56-e14220bd6819 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:29.348650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa3c19-b7b1-4d0f-ad56-e14220bd6819 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:29.360426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa3c19-b7b1-4d0f-ad56-e14220bd6819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-28fa3c19-b7b1-4d0f-ad56-e14220bd6819 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:29.365918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:29.366863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:29.367679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:29.367679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:29.367679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:29.367679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:29.367679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:29.367679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:29.367679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:29.367679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:29.367679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:29.367679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:29.367679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:05:29.367679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:29.367679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 173/350] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:29.368411 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa3c19-b7b1-4d0f-ad56-e14220bd6819 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:29.399569 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc74aae-55c2-40dd-8035-3608de78f952 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:29.412592 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:29.440588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472a25f-40f2-43e2-95e9-ec1a66d623e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:29.441235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472a25f-40f2-43e2-95e9-ec1a66d623e9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:29.441517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472a25f-40f2-43e2-95e9-ec1a66d623e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:29.441791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472a25f-40f2-43e2-95e9-ec1a66d623e9 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:29.443187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472a25f-40f2-43e2-95e9-ec1a66d623e9 None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:29.444669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472a25f-40f2-43e2-95e9-ec1a66d623e9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:29.446659 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 178/351] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:29 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:29.447970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1934b-7e5e-44a1-a3da-abc5d7ea82b8 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:29.448321 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1934b-7e5e-44a1-a3da-abc5d7ea82b8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:29.448600 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1934b-7e5e-44a1-a3da-abc5d7ea82b8 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:29.450059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 179/352] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:29.450769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1934b-7e5e-44a1-a3da-abc5d7ea82b8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:29.482397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:29.486170 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa3c19-b7b1-4d0f-ad56-e14220bd6819 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:29.531468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1261f7-f662-4597-9cc9-4c4f49686981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:29.532273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1261f7-f662-4597-9cc9-4c4f49686981 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:29.532587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1261f7-f662-4597-9cc9-4c4f49686981 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:29.533072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1261f7-f662-4597-9cc9-4c4f49686981 None admin] PATH_INFO: `/v3/groups/f52913c455004781af3e4524f4dbce9d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:29.544834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1261f7-f662-4597-9cc9-4c4f49686981 None admin] RBAC: Authorizing `identity:get_group(group_id=f52913c455004781af3e4524f4dbce9d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:29.546562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1261f7-f662-4597-9cc9-4c4f49686981 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:29.548440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 174/353] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:29 2022] GET /identity/v3/groups/f52913c455004781af3e4524f4dbce9d => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:29.549231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1261f7-f662-4597-9cc9-4c4f49686981 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:29.598100 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1934b-7e5e-44a1-a3da-abc5d7ea82b8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:29.662856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:29.664582 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02168e8c-9379-46be-ba6e-3c2781d4e60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:29.665637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02168e8c-9379-46be-ba6e-3c2781d4e60c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:29.665920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02168e8c-9379-46be-ba6e-3c2781d4e60c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:29.666231 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02168e8c-9379-46be-ba6e-3c2781d4e60c None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:29.666920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02168e8c-9379-46be-ba6e-3c2781d4e60c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:29.667242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02168e8c-9379-46be-ba6e-3c2781d4e60c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:29.668821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02168e8c-9379-46be-ba6e-3c2781d4e60c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:29.678991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 180/354] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:29.684372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c059f0-1f47-498c-9790-db9e7a3f315a None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:29.684736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c059f0-1f47-498c-9790-db9e7a3f315a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:29.685024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1261f7-f662-4597-9cc9-4c4f49686981 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:29.685131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c059f0-1f47-498c-9790-db9e7a3f315a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:29.712950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-48c059f0-1f47-498c-9790-db9e7a3f315a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:29.714898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-48c059f0-1f47-498c-9790-db9e7a3f315a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:29.735283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a9f3e-ed42-4540-9c2d-5df39f826d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:29.736197 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a9f3e-ed42-4540-9c2d-5df39f826d1c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:29.736323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a9f3e-ed42-4540-9c2d-5df39f826d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:29.736608 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a9f3e-ed42-4540-9c2d-5df39f826d1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:29.747864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a9f3e-ed42-4540-9c2d-5df39f826d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-bb1a9f3e-ed42-4540-9c2d-5df39f826d1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:29.753574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:29.754665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:29.755460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:29.755460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:29.755460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:29.755460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:29.755460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:29.755460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:29.755460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:29.755460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:29.755460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:29.755460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:29.755460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:29.755460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:29.755460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 175/355] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:29.756122 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a9f3e-ed42-4540-9c2d-5df39f826d1c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:29.795122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 181/356] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:29.796987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c059f0-1f47-498c-9790-db9e7a3f315a None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:29.816708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:29.886791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a9f3e-ed42-4540-9c2d-5df39f826d1c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:29.931501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c059f0-1f47-498c-9790-db9e7a3f315a None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:29.933961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff61527-58fb-4499-aeb9-9b408d3e3d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:29.934683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff61527-58fb-4499-aeb9-9b408d3e3d8b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:29.935041 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff61527-58fb-4499-aeb9-9b408d3e3d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:29.935397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff61527-58fb-4499-aeb9-9b408d3e3d8b None admin] PATH_INFO: `/v3/groups/f52913c455004781af3e4524f4dbce9d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:29.938976 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff61527-58fb-4499-aeb9-9b408d3e3d8b None admin] RBAC: Authorizing `identity:get_group(group_id=f52913c455004781af3e4524f4dbce9d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:29.939637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff61527-58fb-4499-aeb9-9b408d3e3d8b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:29.941285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 176/357] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:29 2022] GET /identity/v3/groups/f52913c455004781af3e4524f4dbce9d => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:29.942082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff61527-58fb-4499-aeb9-9b408d3e3d8b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:29.966636 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:29.990082 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df0722-febd-439b-9d02-9c6216265a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:29.990703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df0722-febd-439b-9d02-9c6216265a4c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:29.990983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df0722-febd-439b-9d02-9c6216265a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:29.991257 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df0722-febd-439b-9d02-9c6216265a4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:29.992101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df0722-febd-439b-9d02-9c6216265a4c None admin] RBAC: Adding query filter params (group.id=f52913c455004781af3e4524f4dbce9d, role.id=4f10622fcf4c4f708cd6b652d886ba3e, scope.project.id=6d01d0d61d804d4393d51b2d191f652d) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:29.992346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df0722-febd-439b-9d02-9c6216265a4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:29.993785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df0722-febd-439b-9d02-9c6216265a4c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:30.002792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 182/358] 10.222.0.85 () {56 vars in 1376 bytes} [Mon Aug 29 20:05:29 2022] GET /identity/v3/role_assignments?group.id=f52913c455004781af3e4524f4dbce9d&scope.project.id=6d01d0d61d804d4393d51b2d191f652d&role.id=4f10622fcf4c4f708cd6b652d886ba3e => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:30.004028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182d59c-3171-479c-9bfd-597f298955c6 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:30.004339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182d59c-3171-479c-9bfd-597f298955c6 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:30.004611 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182d59c-3171-479c-9bfd-597f298955c6 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:30.005951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 183/359] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:30.006641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182d59c-3171-479c-9bfd-597f298955c6 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:30.082727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff61527-58fb-4499-aeb9-9b408d3e3d8b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:30.125236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:30.133365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f877f-7279-49ee-b2d2-e3ff3fcbb2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:30.134313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f877f-7279-49ee-b2d2-e3ff3fcbb2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:30.134767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f877f-7279-49ee-b2d2-e3ff3fcbb2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:30.135179 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f877f-7279-49ee-b2d2-e3ff3fcbb2e1 None admin] PATH_INFO: `/v3/users/b4f9aecf93be4076a7c301a3c9bc509f` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:30.147859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f877f-7279-49ee-b2d2-e3ff3fcbb2e1 None admin] RBAC: Authorizing `identity:get_user(user_id=b4f9aecf93be4076a7c301a3c9bc509f)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:30.147859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f877f-7279-49ee-b2d2-e3ff3fcbb2e1 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:30.147859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 177/360] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:05:29 2022] GET /identity/v3/users/b4f9aecf93be4076a7c301a3c9bc509f => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:30.147859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f877f-7279-49ee-b2d2-e3ff3fcbb2e1 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:30.158459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182d59c-3171-479c-9bfd-597f298955c6 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:30.207640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c90d7e-3487-4edf-b43b-cc46fff6db04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:30.208693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c90d7e-3487-4edf-b43b-cc46fff6db04 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:30.209572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c90d7e-3487-4edf-b43b-cc46fff6db04 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:30.210091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c90d7e-3487-4edf-b43b-cc46fff6db04 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:30.211292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c90d7e-3487-4edf-b43b-cc46fff6db04 None admin] RBAC: Adding query filter params (name=member) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:30.211758 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c90d7e-3487-4edf-b43b-cc46fff6db04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:30.213205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c90d7e-3487-4edf-b43b-cc46fff6db04 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:30.223208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 184/361] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:30.225080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d398a1d-38f5-405c-8eca-b7335bec1300 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:30.225392 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d398a1d-38f5-405c-8eca-b7335bec1300 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:30.225662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d398a1d-38f5-405c-8eca-b7335bec1300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:30.244455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7d398a1d-38f5-405c-8eca-b7335bec1300 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:30.245569 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7d398a1d-38f5-405c-8eca-b7335bec1300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:30.263131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:30.286159 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f877f-7279-49ee-b2d2-e3ff3fcbb2e1 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:30.325460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 185/362] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:30.326728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d398a1d-38f5-405c-8eca-b7335bec1300 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:30.326848 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dc9fc-2e6b-41b0-93b4-baf5813a2175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:30.327354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dc9fc-2e6b-41b0-93b4-baf5813a2175 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:30.327636 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dc9fc-2e6b-41b0-93b4-baf5813a2175 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:30.327921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dc9fc-2e6b-41b0-93b4-baf5813a2175 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:30.328670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26dc9fc-2e6b-41b0-93b4-baf5813a2175 None admin] RBAC: Adding query filter params (group.id=f52913c455004781af3e4524f4dbce9d, role.id=f1d8af7d158e4e02a6e77492178db259, scope.project.id=6d01d0d61d804d4393d51b2d191f652d) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:30.328982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26dc9fc-2e6b-41b0-93b4-baf5813a2175 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:30.330788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26dc9fc-2e6b-41b0-93b4-baf5813a2175 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:30.346331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 178/363] 10.222.0.85 () {56 vars in 1376 bytes} [Mon Aug 29 20:05:30 2022] GET /identity/v3/role_assignments?group.id=f52913c455004781af3e4524f4dbce9d&scope.project.id=6d01d0d61d804d4393d51b2d191f652d&role.id=f1d8af7d158e4e02a6e77492178db259 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:30.346331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dc9fc-2e6b-41b0-93b4-baf5813a2175 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:30.495312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dc9fc-2e6b-41b0-93b4-baf5813a2175 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:30.500701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:30.559510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fae53-06ec-4986-9e8b-c06044cf3f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:30.560136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fae53-06ec-4986-9e8b-c06044cf3f1d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:30.560440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fae53-06ec-4986-9e8b-c06044cf3f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:30.560730 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fae53-06ec-4986-9e8b-c06044cf3f1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:30.561447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fae53-06ec-4986-9e8b-c06044cf3f1d None admin] RBAC: Adding query filter params (role.id=7da1608958c94a6dbb98eaf955981d2d, user.id=b4f9aecf93be4076a7c301a3c9bc509f, scope.system=all) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:30.561755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fae53-06ec-4986-9e8b-c06044cf3f1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:30.563212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fae53-06ec-4986-9e8b-c06044cf3f1d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:30.574823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d398a1d-38f5-405c-8eca-b7335bec1300 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:30.584786 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 179/364] 10.222.0.85 () {56 vars in 1308 bytes} [Mon Aug 29 20:05:30 2022] GET /identity/v3/role_assignments?user.id=b4f9aecf93be4076a7c301a3c9bc509f&scope.system=all&role.id=7da1608958c94a6dbb98eaf955981d2d => generated 510 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:30.586058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fae53-06ec-4986-9e8b-c06044cf3f1d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:30.589540 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:30.647840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab76b7-3640-41e1-84dc-df906c1d0f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:30.649363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab76b7-3640-41e1-84dc-df906c1d0f09 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:30.649363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab76b7-3640-41e1-84dc-df906c1d0f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:30.649363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab76b7-3640-41e1-84dc-df906c1d0f09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:30.675715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab76b7-3640-41e1-84dc-df906c1d0f09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-feab76b7-3640-41e1-84dc-df906c1d0f09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:30.681264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:30.682434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:30.683465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:30.683465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:30.683465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:30.683465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:30.683465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:30.683465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:30.683465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:30.683465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:30.683465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:30.683465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:30.683465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:30.683465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:30.685472 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 186/365] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:05:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:30.691403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab76b7-3640-41e1-84dc-df906c1d0f09 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:30.734876 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fae53-06ec-4986-9e8b-c06044cf3f1d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:30.781157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e88be3-572d-44cd-9d47-a4b438819714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:30.782357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e88be3-572d-44cd-9d47-a4b438819714 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:30.783337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e88be3-572d-44cd-9d47-a4b438819714 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:30.783767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e88be3-572d-44cd-9d47-a4b438819714 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:30.867303 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e88be3-572d-44cd-9d47-a4b438819714 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:30.869378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e88be3-572d-44cd-9d47-a4b438819714 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:30.873247 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:30.886335 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab76b7-3640-41e1-84dc-df906c1d0f09 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:30.937267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e9c6a-db4d-4a30-9e04-0667d8a23206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:30.937917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e9c6a-db4d-4a30-9e04-0667d8a23206 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:30.938193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e9c6a-db4d-4a30-9e04-0667d8a23206 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:30.938458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e9c6a-db4d-4a30-9e04-0667d8a23206 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:30.944699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e9c6a-db4d-4a30-9e04-0667d8a23206 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:30.945034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e9c6a-db4d-4a30-9e04-0667d8a23206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:30.946560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e9c6a-db4d-4a30-9e04-0667d8a23206 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:30.948503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:30.958909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 187/366] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:30.961252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e9c6a-db4d-4a30-9e04-0667d8a23206 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:31.000023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 180/367] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:31.001660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e88be3-572d-44cd-9d47-a4b438819714 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:31.081560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e9c6a-db4d-4a30-9e04-0667d8a23206 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:31.125835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09718e72-7c2e-4d2c-8e01-a181ae954595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:31.127400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09718e72-7c2e-4d2c-8e01-a181ae954595 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:31.127934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09718e72-7c2e-4d2c-8e01-a181ae954595 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:31.128363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09718e72-7c2e-4d2c-8e01-a181ae954595 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:31.135629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e88be3-572d-44cd-9d47-a4b438819714 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:31.138300 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:31.143483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09718e72-7c2e-4d2c-8e01-a181ae954595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-09718e72-7c2e-4d2c-8e01-a181ae954595 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:31.150781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:31.151653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:31.152496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:31.152496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:31.152496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:31.152496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:31.152496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:31.152496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:31.152496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:31.152496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:31.152496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:31.152496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:31.152496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:31.152496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:31.152496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 188/368] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:05:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:31.153680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09718e72-7c2e-4d2c-8e01-a181ae954595 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:31.187002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95018903-e9ee-4009-8182-0dc393889240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:31.187754 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95018903-e9ee-4009-8182-0dc393889240 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:31.188034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95018903-e9ee-4009-8182-0dc393889240 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:31.188372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95018903-e9ee-4009-8182-0dc393889240 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:31.217205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:31.274750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95018903-e9ee-4009-8182-0dc393889240 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:31.276621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95018903-e9ee-4009-8182-0dc393889240 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:31.281157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09718e72-7c2e-4d2c-8e01-a181ae954595 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:31.323652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adc372-82fe-46cc-b582-366215c85b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:31.324817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adc372-82fe-46cc-b582-366215c85b9f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:31.325398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adc372-82fe-46cc-b582-366215c85b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:31.325905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adc372-82fe-46cc-b582-366215c85b9f None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:31.327396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73adc372-82fe-46cc-b582-366215c85b9f None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:31.329916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73adc372-82fe-46cc-b582-366215c85b9f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:31.332128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 189/369] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:31 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:31.333350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adc372-82fe-46cc-b582-366215c85b9f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:31.398784 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 181/370] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:31.399686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95018903-e9ee-4009-8182-0dc393889240 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:31.407691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:31.467205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adc372-82fe-46cc-b582-366215c85b9f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:31.492156 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:31.515374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ebd02-4b45-43b6-86c3-61b9f4c34dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:31.517387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ebd02-4b45-43b6-86c3-61b9f4c34dba None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:31.517686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ebd02-4b45-43b6-86c3-61b9f4c34dba None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:31.517957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ebd02-4b45-43b6-86c3-61b9f4c34dba None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:31.518720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ebd02-4b45-43b6-86c3-61b9f4c34dba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:31.519041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ebd02-4b45-43b6-86c3-61b9f4c34dba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:31.520495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ebd02-4b45-43b6-86c3-61b9f4c34dba None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:31.532090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 190/371] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:31.534495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ebd02-4b45-43b6-86c3-61b9f4c34dba None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:31.555004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95018903-e9ee-4009-8182-0dc393889240 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:31.601366 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61336dd-d59e-4380-8ecb-d0ac28988d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:31.601912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61336dd-d59e-4380-8ecb-d0ac28988d67 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:31.602201 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61336dd-d59e-4380-8ecb-d0ac28988d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:31.602485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61336dd-d59e-4380-8ecb-d0ac28988d67 None admin] PATH_INFO: `/v3/groups/f52913c455004781af3e4524f4dbce9d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:31.604273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61336dd-d59e-4380-8ecb-d0ac28988d67 None admin] RBAC: Authorizing `identity:get_group(group_id=f52913c455004781af3e4524f4dbce9d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:31.605598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61336dd-d59e-4380-8ecb-d0ac28988d67 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:31.607164 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 182/372] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:31 2022] GET /identity/v3/groups/f52913c455004781af3e4524f4dbce9d => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:31.608055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61336dd-d59e-4380-8ecb-d0ac28988d67 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:31.685733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ebd02-4b45-43b6-86c3-61b9f4c34dba None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:31.701614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:31.731705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117bf53-23d6-49fa-9569-042c0792e96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:31.732396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117bf53-23d6-49fa-9569-042c0792e96f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:31.732672 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117bf53-23d6-49fa-9569-042c0792e96f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:31.732941 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117bf53-23d6-49fa-9569-042c0792e96f None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:31.734084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117bf53-23d6-49fa-9569-042c0792e96f None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:31.735623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117bf53-23d6-49fa-9569-042c0792e96f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:31.737377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 191/373] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:31 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:31.738147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117bf53-23d6-49fa-9569-042c0792e96f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:31.745822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61336dd-d59e-4380-8ecb-d0ac28988d67 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:31.784658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c5266-a9b0-4d23-a702-8bb912c9edde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:31.785676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c5266-a9b0-4d23-a702-8bb912c9edde None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:31.786223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c5266-a9b0-4d23-a702-8bb912c9edde None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:31.786711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c5266-a9b0-4d23-a702-8bb912c9edde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:31.791769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:31.847084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117bf53-23d6-49fa-9569-042c0792e96f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:31.861487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c5266-a9b0-4d23-a702-8bb912c9edde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:31.863357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c5266-a9b0-4d23-a702-8bb912c9edde None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:31.885897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbbfdf-01dc-475d-9a19-8b7fb4ab9871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:31.886513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbbfdf-01dc-475d-9a19-8b7fb4ab9871 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:31.886801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbbfdf-01dc-475d-9a19-8b7fb4ab9871 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:31.887090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbbfdf-01dc-475d-9a19-8b7fb4ab9871 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:31.887841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbbfdf-01dc-475d-9a19-8b7fb4ab9871 None admin] RBAC: Adding query filter params (group.id=f52913c455004781af3e4524f4dbce9d, role.id=4f10622fcf4c4f708cd6b652d886ba3e, scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:31.888186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbbfdf-01dc-475d-9a19-8b7fb4ab9871 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:31.889658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbbfdf-01dc-475d-9a19-8b7fb4ab9871 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:31.896741 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 192/374] 10.222.0.85 () {56 vars in 1376 bytes} [Mon Aug 29 20:05:31 2022] GET /identity/v3/role_assignments?group.id=f52913c455004781af3e4524f4dbce9d&scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24&role.id=4f10622fcf4c4f708cd6b652d886ba3e => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:31.897390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbbfdf-01dc-475d-9a19-8b7fb4ab9871 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:31.944613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:31.980750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 183/375] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:31.987707 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c5266-a9b0-4d23-a702-8bb912c9edde None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:32.064261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbbfdf-01dc-475d-9a19-8b7fb4ab9871 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:32.082683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:32.103574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c5266-a9b0-4d23-a702-8bb912c9edde None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:32.125689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b995c4a-b372-467f-a479-6fe288ad60ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:32.126330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b995c4a-b372-467f-a479-6fe288ad60ec None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:32.126628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b995c4a-b372-467f-a479-6fe288ad60ec None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:32.126911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b995c4a-b372-467f-a479-6fe288ad60ec None admin] PATH_INFO: `/v3/groups/f52913c455004781af3e4524f4dbce9d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:32.130336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b995c4a-b372-467f-a479-6fe288ad60ec None admin] RBAC: Authorizing `identity:get_group(group_id=f52913c455004781af3e4524f4dbce9d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:32.131830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b995c4a-b372-467f-a479-6fe288ad60ec None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:32.133965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 193/376] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:31 2022] GET /identity/v3/groups/f52913c455004781af3e4524f4dbce9d => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:32.139978 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b995c4a-b372-467f-a479-6fe288ad60ec None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:32.153137 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21215dd4-512f-4af3-af5c-aee566dba104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:32.154176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21215dd4-512f-4af3-af5c-aee566dba104 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:32.154671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21215dd4-512f-4af3-af5c-aee566dba104 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:32.155149 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21215dd4-512f-4af3-af5c-aee566dba104 None admin] PATH_INFO: `/v3/projects/e08700a8f1f34f3488594693a51c7bb4` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:32.157670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21215dd4-512f-4af3-af5c-aee566dba104 None admin] RBAC: Authorizing `identity:get_project(project_id=e08700a8f1f34f3488594693a51c7bb4)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:32.159996 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21215dd4-512f-4af3-af5c-aee566dba104 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:32.165213 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 184/377] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:31 2022] GET /identity/v3/projects/e08700a8f1f34f3488594693a51c7bb4 => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:32.169049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21215dd4-512f-4af3-af5c-aee566dba104 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:32.240592 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:32.344694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b995c4a-b372-467f-a479-6fe288ad60ec None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:32.350356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21215dd4-512f-4af3-af5c-aee566dba104 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:32.387836 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ffdc5-b36c-4391-ae35-4ffec085f7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:32.388413 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ffdc5-b36c-4391-ae35-4ffec085f7ab None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:32.388696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ffdc5-b36c-4391-ae35-4ffec085f7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:32.388966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ffdc5-b36c-4391-ae35-4ffec085f7ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:32.389709 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ffdc5-b36c-4391-ae35-4ffec085f7ab None admin] RBAC: Adding query filter params (group.id=f52913c455004781af3e4524f4dbce9d, role.id=f1d8af7d158e4e02a6e77492178db259, scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:32.390013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ffdc5-b36c-4391-ae35-4ffec085f7ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:32.391689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ffdc5-b36c-4391-ae35-4ffec085f7ab None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:32.399622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d33d2-04a3-4fba-8c42-7b134c4dce27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:32.399822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 194/378] 10.222.0.85 () {56 vars in 1376 bytes} [Mon Aug 29 20:05:32 2022] GET /identity/v3/role_assignments?group.id=f52913c455004781af3e4524f4dbce9d&scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24&role.id=f1d8af7d158e4e02a6e77492178db259 => generated 260 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:32.400907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d33d2-04a3-4fba-8c42-7b134c4dce27 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:32.401366 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d33d2-04a3-4fba-8c42-7b134c4dce27 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:32.401485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9ea14-3b0e-4142-aaff-f0ee4a238d8f None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:32.401757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9ea14-3b0e-4142-aaff-f0ee4a238d8f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:32.402039 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9ea14-3b0e-4142-aaff-f0ee4a238d8f None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:32.402169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d33d2-04a3-4fba-8c42-7b134c4dce27 None admin] PATH_INFO: `/v3/groups/67e9612ed98343fabf0bc5f9c55fc83f` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:32.408301 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 195/379] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:32.411710 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2ee7f-63dc-4c59-aaf1-c97816fd626d None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:32.411991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2ee7f-63dc-4c59-aaf1-c97816fd626d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:32.412381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2ee7f-63dc-4c59-aaf1-c97816fd626d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:32.415297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d33d2-04a3-4fba-8c42-7b134c4dce27 None admin] RBAC: Authorizing `identity:get_group(group_id=67e9612ed98343fabf0bc5f9c55fc83f)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:32.417040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d33d2-04a3-4fba-8c42-7b134c4dce27 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:32.418879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 185/380] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:32 2022] GET /identity/v3/groups/67e9612ed98343fabf0bc5f9c55fc83f => generated 229 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:32.425097 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d33d2-04a3-4fba-8c42-7b134c4dce27 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:32.434759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-bbf2ee7f-63dc-4c59-aaf1-c97816fd626d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:32.436020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-bbf2ee7f-63dc-4c59-aaf1-c97816fd626d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:32.484373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:32.517540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 196/381] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:32.559810 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d33d2-04a3-4fba-8c42-7b134c4dce27 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:32.564518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:32.605624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdacd1e8-8978-45d8-90a2-996587a21894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:32.606259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacd1e8-8978-45d8-90a2-996587a21894 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:32.606542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacd1e8-8978-45d8-90a2-996587a21894 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:32.606818 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacd1e8-8978-45d8-90a2-996587a21894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:32.607637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdacd1e8-8978-45d8-90a2-996587a21894 None admin] RBAC: Adding query filter params (group.id=67e9612ed98343fabf0bc5f9c55fc83f, role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=e08700a8f1f34f3488594693a51c7bb4) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:32.607952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdacd1e8-8978-45d8-90a2-996587a21894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:32.609521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdacd1e8-8978-45d8-90a2-996587a21894 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:32.621144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 186/382] 10.222.0.85 () {56 vars in 1376 bytes} [Mon Aug 29 20:05:32 2022] GET /identity/v3/role_assignments?group.id=67e9612ed98343fabf0bc5f9c55fc83f&scope.project.id=e08700a8f1f34f3488594693a51c7bb4&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:32.845350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2ee7f-63dc-4c59-aaf1-c97816fd626d None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:32.945372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03548f84-e4b8-4689-93c5-99fa287c5d15 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:32.945660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03548f84-e4b8-4689-93c5-99fa287c5d15 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:32.945944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03548f84-e4b8-4689-93c5-99fa287c5d15 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:32.947546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 187/383] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:32.957940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edd264-c600-417f-9815-37773ad22b01 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:32.958225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edd264-c600-417f-9815-37773ad22b01 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:32.958507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edd264-c600-417f-9815-37773ad22b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:32.969270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2ee7f-63dc-4c59-aaf1-c97816fd626d None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:32.995569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d5edd264-c600-417f-9815-37773ad22b01 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:32.996849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d5edd264-c600-417f-9815-37773ad22b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:33.018581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e0e3c-91dc-46be-8bd8-0b90d5d577ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:33.019265 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e0e3c-91dc-46be-8bd8-0b90d5d577ed None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:33.019559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e0e3c-91dc-46be-8bd8-0b90d5d577ed None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:33.019866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e0e3c-91dc-46be-8bd8-0b90d5d577ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:33.041713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e0e3c-91dc-46be-8bd8-0b90d5d577ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-e20e0e3c-91dc-46be-8bd8-0b90d5d577ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:33.049696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:33.050628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:33.051431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:33.051431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:33.051431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:33.051431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:33.051431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:33.051431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:33.051431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:33.051431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:33.051431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:33.051431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:33.051431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:33.051431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:33.052598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 197/384] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:33.059748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e0e3c-91dc-46be-8bd8-0b90d5d577ed None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:33.107608 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:33.144145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 188/385] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:33.185559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e0e3c-91dc-46be-8bd8-0b90d5d577ed None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:33.224287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766ffcc-82a5-4ce7-a46f-cfd9aded117f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:33.225183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766ffcc-82a5-4ce7-a46f-cfd9aded117f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:33.225463 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766ffcc-82a5-4ce7-a46f-cfd9aded117f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:33.225732 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766ffcc-82a5-4ce7-a46f-cfd9aded117f None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:33.226496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766ffcc-82a5-4ce7-a46f-cfd9aded117f None admin] RBAC: Adding query filter params (name=member) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:33.227020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766ffcc-82a5-4ce7-a46f-cfd9aded117f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:33.229017 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766ffcc-82a5-4ce7-a46f-cfd9aded117f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:33.238680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 198/386] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:33.244462 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edd264-c600-417f-9815-37773ad22b01 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:33.377492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:33.385534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edd264-c600-417f-9815-37773ad22b01 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:33.428576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81565c89-3eef-40de-aa4e-af116d4cb40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:33.429221 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81565c89-3eef-40de-aa4e-af116d4cb40d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:33.429504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81565c89-3eef-40de-aa4e-af116d4cb40d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:33.429776 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81565c89-3eef-40de-aa4e-af116d4cb40d None admin] PATH_INFO: `/v3/groups/f52913c455004781af3e4524f4dbce9d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:33.430904 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81565c89-3eef-40de-aa4e-af116d4cb40d None admin] RBAC: Authorizing `identity:get_group(group_id=f52913c455004781af3e4524f4dbce9d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:33.432370 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81565c89-3eef-40de-aa4e-af116d4cb40d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:33.434244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 189/387] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:33 2022] GET /identity/v3/groups/f52913c455004781af3e4524f4dbce9d => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:33.440970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766ffcc-82a5-4ce7-a46f-cfd9aded117f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:33.566538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:33.576950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766ffcc-82a5-4ce7-a46f-cfd9aded117f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:33.616253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81565c89-3eef-40de-aa4e-af116d4cb40d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:33.620026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f0ff4-5275-4a2a-beff-052c0df87be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:33.620707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f0ff4-5275-4a2a-beff-052c0df87be4 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:33.620985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f0ff4-5275-4a2a-beff-052c0df87be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:33.621312 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f0ff4-5275-4a2a-beff-052c0df87be4 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:33.622520 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f0ff4-5275-4a2a-beff-052c0df87be4 None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:33.624021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f0ff4-5275-4a2a-beff-052c0df87be4 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:33.625672 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 199/388] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:33 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:33.638247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f0ff4-5275-4a2a-beff-052c0df87be4 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:33.743003 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81565c89-3eef-40de-aa4e-af116d4cb40d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:33.783793 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dda17-36a5-48df-9120-5d8f34adc94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:33.784463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dda17-36a5-48df-9120-5d8f34adc94c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:33.784740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dda17-36a5-48df-9120-5d8f34adc94c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:33.785013 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dda17-36a5-48df-9120-5d8f34adc94c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:33.795583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36dda17-36a5-48df-9120-5d8f34adc94c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-f36dda17-36a5-48df-9120-5d8f34adc94c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:33.801789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:33.802681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:33.803534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:33.803534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:33.803534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:33.803534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:33.803534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:33.803534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:33.803534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:33.803534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:33.803534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:33.803534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:33.803534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:33.803534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:33.803534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 190/389] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:05:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:33.809806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dda17-36a5-48df-9120-5d8f34adc94c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:33.810610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f0ff4-5275-4a2a-beff-052c0df87be4 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:33.840159 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:33.886445 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb46e7a-1b5f-4695-b848-23bf7c769655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:33.887539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb46e7a-1b5f-4695-b848-23bf7c769655 None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:33.888027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb46e7a-1b5f-4695-b848-23bf7c769655 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:33.888597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb46e7a-1b5f-4695-b848-23bf7c769655 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d/groups/f52913c455004781af3e4524f4dbce9d/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:33.893534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb46e7a-1b5f-4695-b848-23bf7c769655 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d01d0d61d804d4393d51b2d191f652d, group_id=f52913c455004781af3e4524f4dbce9d, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:33.895351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb46e7a-1b5f-4695-b848-23bf7c769655 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:33.945133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dda17-36a5-48df-9120-5d8f34adc94c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:33.985026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db15bc-e0b6-4595-9f68-0d7810002219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:33.986203 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db15bc-e0b6-4595-9f68-0d7810002219 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:33.986607 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db15bc-e0b6-4595-9f68-0d7810002219 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:33.987029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db15bc-e0b6-4595-9f68-0d7810002219 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:33.987867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db15bc-e0b6-4595-9f68-0d7810002219 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:33.988211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db15bc-e0b6-4595-9f68-0d7810002219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:33.989621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db15bc-e0b6-4595-9f68-0d7810002219 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:33.996034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 200/390] 10.222.0.85 () {58 vars in 1442 bytes} [Mon Aug 29 20:05:33 2022] PUT /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d/groups/f52913c455004781af3e4524f4dbce9d/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:05:34.001966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 191/391] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:34.007586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb46e7a-1b5f-4695-b848-23bf7c769655 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:34.055101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:34.152830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb46e7a-1b5f-4695-b848-23bf7c769655 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:34.191733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac59d1-18b9-4134-9524-c9a4db27c598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:34.192446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac59d1-18b9-4134-9524-c9a4db27c598 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:34.192780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac59d1-18b9-4134-9524-c9a4db27c598 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:34.193051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac59d1-18b9-4134-9524-c9a4db27c598 None admin] PATH_INFO: `/v3/groups/f52913c455004781af3e4524f4dbce9d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:34.194202 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac59d1-18b9-4134-9524-c9a4db27c598 None admin] RBAC: Authorizing `identity:get_group(group_id=f52913c455004781af3e4524f4dbce9d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:34.195695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac59d1-18b9-4134-9524-c9a4db27c598 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:34.197330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 201/392] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:34 2022] GET /identity/v3/groups/f52913c455004781af3e4524f4dbce9d => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:34.203021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db15bc-e0b6-4595-9f68-0d7810002219 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:34.344420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:34.374198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db15bc-e0b6-4595-9f68-0d7810002219 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:34.457427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81c9f7-1723-412e-b587-573ab65c03d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:34.458460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81c9f7-1723-412e-b587-573ab65c03d3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:34.458984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81c9f7-1723-412e-b587-573ab65c03d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:34.459493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81c9f7-1723-412e-b587-573ab65c03d3 None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:34.461582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81c9f7-1723-412e-b587-573ab65c03d3 None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:34.464478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81c9f7-1723-412e-b587-573ab65c03d3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:34.468111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 192/393] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:34 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:34.473939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac59d1-18b9-4134-9524-c9a4db27c598 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:34.601555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:34.607412 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e30a7-8e2a-4125-8d24-77fc627aa33c None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:34.607702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e30a7-8e2a-4125-8d24-77fc627aa33c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:34.607996 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e30a7-8e2a-4125-8d24-77fc627aa33c None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:34.609924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 193/394] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:34.612624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac59d1-18b9-4134-9524-c9a4db27c598 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:34.618569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb6db8-0cf5-45ff-99bc-b4b545fb1ef0 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:34.618569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb6db8-0cf5-45ff-99bc-b4b545fb1ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:34.618569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb6db8-0cf5-45ff-99bc-b4b545fb1ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:34.653900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0705c4c-6e37-4635-af0f-60bda9a75efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:34.654564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0705c4c-6e37-4635-af0f-60bda9a75efb None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:34.654848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0705c4c-6e37-4635-af0f-60bda9a75efb None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:34.655121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0705c4c-6e37-4635-af0f-60bda9a75efb None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d/groups/f52913c455004781af3e4524f4dbce9d/roles/f1d8af7d158e4e02a6e77492178db259` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:34.659176 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0705c4c-6e37-4635-af0f-60bda9a75efb None admin] RBAC: Authorizing `identity:create_grant(project_id=6d01d0d61d804d4393d51b2d191f652d, group_id=f52913c455004781af3e4524f4dbce9d, role_id=f1d8af7d158e4e02a6e77492178db259)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:34.660436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-9ceb6db8-0cf5-45ff-99bc-b4b545fb1ef0 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:34.660679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0705c4c-6e37-4635-af0f-60bda9a75efb None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:34.662026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9ceb6db8-0cf5-45ff-99bc-b4b545fb1ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:34.744475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 202/395] 10.222.0.85 () {58 vars in 1442 bytes} [Mon Aug 29 20:05:34 2022] PUT /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d/groups/f52913c455004781af3e4524f4dbce9d/roles/f1d8af7d158e4e02a6e77492178db259 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:05:34.769291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 194/396] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:34.959770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba0bd-1aa3-4816-b5b8-f1f1345c819e None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:34.960186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba0bd-1aa3-4816-b5b8-f1f1345c819e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:34.960434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba0bd-1aa3-4816-b5b8-f1f1345c819e None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:34.961951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 203/397] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:34.971739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b4304-d8ed-4234-b6dd-34f598f69e5c None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:34.972037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b4304-d8ed-4234-b6dd-34f598f69e5c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:34.972319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b4304-d8ed-4234-b6dd-34f598f69e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:34.991130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-c13b4304-d8ed-4234-b6dd-34f598f69e5c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:34.992376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c13b4304-d8ed-4234-b6dd-34f598f69e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:35.073378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 195/398] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:35.143811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ba0bd-1aa3-4816-b5b8-f1f1345c819e None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:35.251983 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003f0c-9844-429f-85eb-41d21347d303 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:35.252290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003f0c-9844-429f-85eb-41d21347d303 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:35.252565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003f0c-9844-429f-85eb-41d21347d303 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:35.254051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 196/399] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:35.258714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ba0bd-1aa3-4816-b5b8-f1f1345c819e None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:35.259563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb326ed-68c6-4e76-a6dc-3b72091d6cee None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:35.259858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb326ed-68c6-4e76-a6dc-3b72091d6cee None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:35.260141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb326ed-68c6-4e76-a6dc-3b72091d6cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:35.281080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-4bb326ed-68c6-4e76-a6dc-3b72091d6cee None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:35.283532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-4bb326ed-68c6-4e76-a6dc-3b72091d6cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:35.300451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:35.332998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bef22-79ca-49d6-81a8-5908774796ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:35.333551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bef22-79ca-49d6-81a8-5908774796ab None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:35.333831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bef22-79ca-49d6-81a8-5908774796ab None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:35.334102 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bef22-79ca-49d6-81a8-5908774796ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:35.357434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bef22-79ca-49d6-81a8-5908774796ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-9d9bef22-79ca-49d6-81a8-5908774796ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:35.363535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:35.364486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:35.365614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:35.365614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:35.365614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:35.365614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:35.365614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:35.365614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:35.365614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:35.365614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:35.365614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:35.365614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:35.365614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:35.365614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:35.365614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 204/400] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:35.366284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bef22-79ca-49d6-81a8-5908774796ab None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:35.385329 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:35.392513 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 197/401] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:05:35.394448 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb326ed-68c6-4e76-a6dc-3b72091d6cee None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:35.483473 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bef22-79ca-49d6-81a8-5908774796ab None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:35.522201 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb326ed-68c6-4e76-a6dc-3b72091d6cee None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:35.536641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f419f-61c2-4593-ad7c-c45a15ecd1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:35.540480 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f419f-61c2-4593-ad7c-c45a15ecd1aa None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:35.540766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f419f-61c2-4593-ad7c-c45a15ecd1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:35.541040 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f419f-61c2-4593-ad7c-c45a15ecd1aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:35.564783 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f419f-61c2-4593-ad7c-c45a15ecd1aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-e39f419f-61c2-4593-ad7c-c45a15ecd1aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:35.570020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:35.572073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:35.573919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:35.573919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:35.573919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:35.573919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:35.573919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:35.573919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:35.573919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:35.573919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:35.573919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:35.573919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:35.573919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:35.573919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:35.573919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 205/402] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:05:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:35.577173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f419f-61c2-4593-ad7c-c45a15ecd1aa None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:35.584588 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e8366-78aa-4786-87e2-131ef5578b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:35.584969 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e8366-78aa-4786-87e2-131ef5578b01 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:35.585290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e8366-78aa-4786-87e2-131ef5578b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:35.585565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e8366-78aa-4786-87e2-131ef5578b01 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:35.586241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e8366-78aa-4786-87e2-131ef5578b01 None admin] RBAC: Adding query filter params (name=member) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:35.586548 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e8366-78aa-4786-87e2-131ef5578b01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:35.587919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e8366-78aa-4786-87e2-131ef5578b01 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:35.607056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 198/403] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:35.612544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e8366-78aa-4786-87e2-131ef5578b01 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:35.627388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:35.715984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:35.768877 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f419f-61c2-4593-ad7c-c45a15ecd1aa None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:35.792729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e8366-78aa-4786-87e2-131ef5578b01 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:35.818132 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e292dd9-a338-4395-90a0-bf56017af2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:35.819277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e292dd9-a338-4395-90a0-bf56017af2ab None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:35.819683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e292dd9-a338-4395-90a0-bf56017af2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:35.819968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e292dd9-a338-4395-90a0-bf56017af2ab None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:35.820917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e292dd9-a338-4395-90a0-bf56017af2ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:35.821225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e292dd9-a338-4395-90a0-bf56017af2ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:35.823286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e292dd9-a338-4395-90a0-bf56017af2ab None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:35.830061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40741d09-6dfc-4881-a9fc-2163ab09639b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:35.830792 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40741d09-6dfc-4881-a9fc-2163ab09639b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:35.831076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40741d09-6dfc-4881-a9fc-2163ab09639b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:35.831346 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40741d09-6dfc-4881-a9fc-2163ab09639b None admin] PATH_INFO: `/v3/groups/f52913c455004781af3e4524f4dbce9d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:35.832498 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40741d09-6dfc-4881-a9fc-2163ab09639b None admin] RBAC: Authorizing `identity:get_group(group_id=f52913c455004781af3e4524f4dbce9d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:35.833793 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 206/404] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:35.833917 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40741d09-6dfc-4881-a9fc-2163ab09639b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:35.834793 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e292dd9-a338-4395-90a0-bf56017af2ab None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:35.835670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 199/405] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:35 2022] GET /identity/v3/groups/f52913c455004781af3e4524f4dbce9d => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:35.839788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40741d09-6dfc-4881-a9fc-2163ab09639b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:35.927316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:35.961663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40741d09-6dfc-4881-a9fc-2163ab09639b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:35.966536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e292dd9-a338-4395-90a0-bf56017af2ab None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:36.004272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c35b93-17d8-47fd-81aa-6efd3b5f0816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:36.004950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c35b93-17d8-47fd-81aa-6efd3b5f0816 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:36.005258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c35b93-17d8-47fd-81aa-6efd3b5f0816 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:36.005494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c35b93-17d8-47fd-81aa-6efd3b5f0816 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:36.006856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69d5c4-d8f0-4f1f-b5f4-35ba3fa9fba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:36.007433 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:36.007865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69d5c4-d8f0-4f1f-b5f4-35ba3fa9fba1 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:36.008496 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69d5c4-d8f0-4f1f-b5f4-35ba3fa9fba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:36.008921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69d5c4-d8f0-4f1f-b5f4-35ba3fa9fba1 None admin] PATH_INFO: `/v3/groups/f52913c455004781af3e4524f4dbce9d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:36.010533 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69d5c4-d8f0-4f1f-b5f4-35ba3fa9fba1 None admin] RBAC: Authorizing `identity:get_group(group_id=f52913c455004781af3e4524f4dbce9d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:36.012625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69d5c4-d8f0-4f1f-b5f4-35ba3fa9fba1 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:36.015367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 200/406] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:35 2022] GET /identity/v3/groups/f52913c455004781af3e4524f4dbce9d => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:36.016493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69d5c4-d8f0-4f1f-b5f4-35ba3fa9fba1 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:36.024095 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c35b93-17d8-47fd-81aa-6efd3b5f0816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-52c35b93-17d8-47fd-81aa-6efd3b5f0816 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:36.029903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:36.031058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:36.031944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:36.031944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:36.031944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:36.031944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:36.031944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:36.031944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:36.031944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:36.031944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:36.031944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:36.031944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:36.031944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:36.031944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:36.031944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 207/407] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:05:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:36.033193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c35b93-17d8-47fd-81aa-6efd3b5f0816 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:36.142609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c35b93-17d8-47fd-81aa-6efd3b5f0816 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:36.148454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69d5c4-d8f0-4f1f-b5f4-35ba3fa9fba1 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:36.181226 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ce87a-d95b-4cfc-b609-0c35c0f2c1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:36.182050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ce87a-d95b-4cfc-b609-0c35c0f2c1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:36.182449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ce87a-d95b-4cfc-b609-0c35c0f2c1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:36.183106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ce87a-d95b-4cfc-b609-0c35c0f2c1f4 None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:36.184251 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ce87a-d95b-4cfc-b609-0c35c0f2c1f4 None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:36.185834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ce87a-d95b-4cfc-b609-0c35c0f2c1f4 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:36.187885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 208/408] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:36 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:36.189404 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ce87a-d95b-4cfc-b609-0c35c0f2c1f4 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:36.202346 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed0e46-5198-4aa1-8e6e-a2c90d9d0529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:36.203321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed0e46-5198-4aa1-8e6e-a2c90d9d0529 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:36.204215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed0e46-5198-4aa1-8e6e-a2c90d9d0529 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:36.204794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed0e46-5198-4aa1-8e6e-a2c90d9d0529 None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:36.206222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed0e46-5198-4aa1-8e6e-a2c90d9d0529 None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:36.208492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed0e46-5198-4aa1-8e6e-a2c90d9d0529 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:36.211352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 201/409] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:36 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:36.212383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed0e46-5198-4aa1-8e6e-a2c90d9d0529 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:36.230684 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:36.258667 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:36.314403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ce87a-d95b-4cfc-b609-0c35c0f2c1f4 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:36.350501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48301ae9-821a-489f-ad2a-e83f660fce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:36.351553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48301ae9-821a-489f-ad2a-e83f660fce9c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:36.352033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48301ae9-821a-489f-ad2a-e83f660fce9c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:36.352530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48301ae9-821a-489f-ad2a-e83f660fce9c None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:36.353389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48301ae9-821a-489f-ad2a-e83f660fce9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:36.353718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed0e46-5198-4aa1-8e6e-a2c90d9d0529 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:36.354141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48301ae9-821a-489f-ad2a-e83f660fce9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:36.355709 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48301ae9-821a-489f-ad2a-e83f660fce9c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:36.365992 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 209/410] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:36.367124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48301ae9-821a-489f-ad2a-e83f660fce9c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:36.399109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c08be-338c-4cf2-a4f1-f9adbe403434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:36.399703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c08be-338c-4cf2-a4f1-f9adbe403434 None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:36.399985 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c08be-338c-4cf2-a4f1-f9adbe403434 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:36.400632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c08be-338c-4cf2-a4f1-f9adbe403434 None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24/groups/f52913c455004781af3e4524f4dbce9d/roles/f1d8af7d158e4e02a6e77492178db259` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:36.403753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c08be-338c-4cf2-a4f1-f9adbe403434 None admin] RBAC: Authorizing `identity:create_grant(project_id=d120d625e6a84d1c9aa9d1bfab52dd24, group_id=f52913c455004781af3e4524f4dbce9d, role_id=f1d8af7d158e4e02a6e77492178db259)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:36.405596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c08be-338c-4cf2-a4f1-f9adbe403434 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:36.475981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48301ae9-821a-489f-ad2a-e83f660fce9c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:36.502004 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:36.513064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaaa6fc-784b-452a-a536-3af6017c63e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:36.513720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaaa6fc-784b-452a-a536-3af6017c63e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:36.513998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaaa6fc-784b-452a-a536-3af6017c63e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:36.514270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaaa6fc-784b-452a-a536-3af6017c63e7 None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24/groups/f52913c455004781af3e4524f4dbce9d/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:36.516149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaaa6fc-784b-452a-a536-3af6017c63e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d120d625e6a84d1c9aa9d1bfab52dd24, group_id=f52913c455004781af3e4524f4dbce9d, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:36.517642 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaaa6fc-784b-452a-a536-3af6017c63e7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:36.565211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 202/411] 10.222.0.85 () {58 vars in 1442 bytes} [Mon Aug 29 20:05:36 2022] PUT /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24/groups/f52913c455004781af3e4524f4dbce9d/roles/f1d8af7d158e4e02a6e77492178db259 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:05:36.568351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c08be-338c-4cf2-a4f1-f9adbe403434 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:36.660410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 210/412] 10.222.0.85 () {58 vars in 1442 bytes} [Mon Aug 29 20:05:36 2022] PUT /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24/groups/f52913c455004781af3e4524f4dbce9d/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:05:36.662037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:36.665230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871ccac-fdca-405b-a7db-d043bba08cbd None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:36.665851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871ccac-fdca-405b-a7db-d043bba08cbd None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:36.673424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871ccac-fdca-405b-a7db-d043bba08cbd None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:36.673983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 211/413] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:36 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:36.675623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080c08d-6911-42f7-af51-94e73e18aaee None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:36.676591 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080c08d-6911-42f7-af51-94e73e18aaee None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:36.677063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080c08d-6911-42f7-af51-94e73e18aaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:36.722151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7080c08d-6911-42f7-af51-94e73e18aaee None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:36.723530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7080c08d-6911-42f7-af51-94e73e18aaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:36.744585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c08be-338c-4cf2-a4f1-f9adbe403434 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:36.804624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33d9fc-fc3f-4df4-9f54-f27c6c354154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:36.805824 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33d9fc-fc3f-4df4-9f54-f27c6c354154 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:36.806331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33d9fc-fc3f-4df4-9f54-f27c6c354154 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:36.806848 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33d9fc-fc3f-4df4-9f54-f27c6c354154 None admin] PATH_INFO: `/v3/groups/67e9612ed98343fabf0bc5f9c55fc83f` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:36.808399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33d9fc-fc3f-4df4-9f54-f27c6c354154 None admin] RBAC: Authorizing `identity:get_group(group_id=67e9612ed98343fabf0bc5f9c55fc83f)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:36.810156 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33d9fc-fc3f-4df4-9f54-f27c6c354154 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:36.815004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 203/414] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:36 2022] GET /identity/v3/groups/67e9612ed98343fabf0bc5f9c55fc83f => generated 229 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:36.821394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33d9fc-fc3f-4df4-9f54-f27c6c354154 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:36.832834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 212/415] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:36.935074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33d9fc-fc3f-4df4-9f54-f27c6c354154 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:36.975397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170111fe-c57a-4b6c-968f-1c23555b248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:36.976401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170111fe-c57a-4b6c-968f-1c23555b248a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:36.980460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170111fe-c57a-4b6c-968f-1c23555b248a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:36.980460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170111fe-c57a-4b6c-968f-1c23555b248a None admin] PATH_INFO: `/v3/projects/e08700a8f1f34f3488594693a51c7bb4` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:36.980460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170111fe-c57a-4b6c-968f-1c23555b248a None admin] RBAC: Authorizing `identity:get_project(project_id=e08700a8f1f34f3488594693a51c7bb4)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:36.980460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170111fe-c57a-4b6c-968f-1c23555b248a None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:36.981632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 204/416] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:36 2022] GET /identity/v3/projects/e08700a8f1f34f3488594693a51c7bb4 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:36.987555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080c08d-6911-42f7-af51-94e73e18aaee None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:37.129857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080c08d-6911-42f7-af51-94e73e18aaee None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:37.134307 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170111fe-c57a-4b6c-968f-1c23555b248a None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:37.156561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:37.162901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:37.193892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc1a4e-5b93-4103-ade3-0f565c9da0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:37.194936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc1a4e-5b93-4103-ade3-0f565c9da0e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:37.195422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc1a4e-5b93-4103-ade3-0f565c9da0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:37.195900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc1a4e-5b93-4103-ade3-0f565c9da0e2 None admin] PATH_INFO: `/v3/projects/e08700a8f1f34f3488594693a51c7bb4/groups/67e9612ed98343fabf0bc5f9c55fc83f/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:37.198204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc1a4e-5b93-4103-ade3-0f565c9da0e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e08700a8f1f34f3488594693a51c7bb4, group_id=67e9612ed98343fabf0bc5f9c55fc83f, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:37.199902 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc1a4e-5b93-4103-ade3-0f565c9da0e2 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:37.267405 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170111fe-c57a-4b6c-968f-1c23555b248a None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:37.305462 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042f3f4-d1b8-4adf-9bc9-adc5e474c963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:37.306263 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042f3f4-d1b8-4adf-9bc9-adc5e474c963 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:37.306551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042f3f4-d1b8-4adf-9bc9-adc5e474c963 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:37.306915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042f3f4-d1b8-4adf-9bc9-adc5e474c963 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:37.317987 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042f3f4-d1b8-4adf-9bc9-adc5e474c963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-a042f3f4-d1b8-4adf-9bc9-adc5e474c963 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:37.323527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:37.324742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:37.325562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:37.325562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:37.325562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:37.325562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:37.325562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:37.325562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:37.325562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:37.325562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:37.325562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:37.325562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:37.325562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:37.325562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:37.329917 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 205/417] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:37.331184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ba761-e2cb-4df4-a767-415e200d9ff8 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:37.331474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ba761-e2cb-4df4-a767-415e200d9ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:37.331753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ba761-e2cb-4df4-a767-415e200d9ff8 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:37.333210 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 206/418] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:37.333790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ba761-e2cb-4df4-a767-415e200d9ff8 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:37.368759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 213/419] 10.222.0.85 () {58 vars in 1442 bytes} [Mon Aug 29 20:05:36 2022] PUT /identity/v3/projects/e08700a8f1f34f3488594693a51c7bb4/groups/67e9612ed98343fabf0bc5f9c55fc83f/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:05:37.370871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea76364-c9ce-4c8f-92ba-11044732ea38 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:37.373990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea76364-c9ce-4c8f-92ba-11044732ea38 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:37.373990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea76364-c9ce-4c8f-92ba-11044732ea38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:37.414590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-1ea76364-c9ce-4c8f-92ba-11044732ea38 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:37.419206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1ea76364-c9ce-4c8f-92ba-11044732ea38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:37.457707 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:37.540084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ba761-e2cb-4df4-a767-415e200d9ff8 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:37.559281 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 214/420] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:37.583421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4f30a-3300-488a-823e-cad5746df43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:37.588188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f30a-3300-488a-823e-cad5746df43f None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:37.588188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f30a-3300-488a-823e-cad5746df43f None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:37.588188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f30a-3300-488a-823e-cad5746df43f None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:37.588188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4f30a-3300-488a-823e-cad5746df43f None admin] RBAC: Adding query filter params (name=member) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:37.588188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4f30a-3300-488a-823e-cad5746df43f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:37.588188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4f30a-3300-488a-823e-cad5746df43f None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:37.597101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 207/421] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:37.602909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea76364-c9ce-4c8f-92ba-11044732ea38 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:37.715792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea76364-c9ce-4c8f-92ba-11044732ea38 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:37.718645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:37.752678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccf549-80b4-40bf-952e-ce8e25f14bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:37.754114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccf549-80b4-40bf-952e-ce8e25f14bab None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:37.754613 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccf549-80b4-40bf-952e-ce8e25f14bab None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:37.755077 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccf549-80b4-40bf-952e-ce8e25f14bab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:37.848672 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccf549-80b4-40bf-952e-ce8e25f14bab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:37.850495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccf549-80b4-40bf-952e-ce8e25f14bab None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:37.969703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 215/422] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:37.971248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4f30a-3300-488a-823e-cad5746df43f None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:37.975974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccf549-80b4-40bf-952e-ce8e25f14bab None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:37.979041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:38.105681 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccf549-80b4-40bf-952e-ce8e25f14bab None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:38.115752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4f30a-3300-488a-823e-cad5746df43f None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:38.142840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92ec97-13fa-4ada-a129-b20368292ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:38.143406 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:38.144670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92ec97-13fa-4ada-a129-b20368292ece None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:38.145111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92ec97-13fa-4ada-a129-b20368292ece None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:38.145538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92ec97-13fa-4ada-a129-b20368292ece None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:38.146942 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92ec97-13fa-4ada-a129-b20368292ece None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:38.148678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92ec97-13fa-4ada-a129-b20368292ece None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:38.151333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 216/423] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:37 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:38.151591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546849d-2e30-469c-84dd-71567708c378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:38.152287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546849d-2e30-469c-84dd-71567708c378 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:38.152568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546849d-2e30-469c-84dd-71567708c378 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:38.152838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546849d-2e30-469c-84dd-71567708c378 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:38.156842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92ec97-13fa-4ada-a129-b20368292ece None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:38.163756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546849d-2e30-469c-84dd-71567708c378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-0546849d-2e30-469c-84dd-71567708c378 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:38.169155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:38.170065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:38.174943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:38.174943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:38.174943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:38.174943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:38.174943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:38.174943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:38.174943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:38.174943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:38.174943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:38.174943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:38.174943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:38.174943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:38.174943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 208/424] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:05:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:05:38.180416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546849d-2e30-469c-84dd-71567708c378 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:38.291300 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546849d-2e30-469c-84dd-71567708c378 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:38.314569 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:38.328538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48edc5f-f93e-481f-b00e-249b3cce52fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:38.329363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48edc5f-f93e-481f-b00e-249b3cce52fe None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:38.329459 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48edc5f-f93e-481f-b00e-249b3cce52fe None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:38.329722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48edc5f-f93e-481f-b00e-249b3cce52fe None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:38.330515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48edc5f-f93e-481f-b00e-249b3cce52fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:38.330824 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48edc5f-f93e-481f-b00e-249b3cce52fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:38.332220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48edc5f-f93e-481f-b00e-249b3cce52fe None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:38.346190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 209/425] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:38.347629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48edc5f-f93e-481f-b00e-249b3cce52fe None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:38.380639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92ec97-13fa-4ada-a129-b20368292ece None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:38.401543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:38.455062 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c8b32-f6b0-424a-a8b0-c6175c8b8597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:38.455703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8b32-f6b0-424a-a8b0-c6175c8b8597 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:38.455982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8b32-f6b0-424a-a8b0-c6175c8b8597 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:38.456281 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8b32-f6b0-424a-a8b0-c6175c8b8597 None admin] PATH_INFO: `/v3/groups/f52913c455004781af3e4524f4dbce9d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:38.457444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c8b32-f6b0-424a-a8b0-c6175c8b8597 None admin] RBAC: Authorizing `identity:get_group(group_id=f52913c455004781af3e4524f4dbce9d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:38.458798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48edc5f-f93e-481f-b00e-249b3cce52fe None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:38.458911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c8b32-f6b0-424a-a8b0-c6175c8b8597 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:38.468670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 217/426] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:38 2022] GET /identity/v3/groups/f52913c455004781af3e4524f4dbce9d => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:38.474347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c8b32-f6b0-424a-a8b0-c6175c8b8597 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:38.494966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e54b-6791-4723-b771-bf1dcdf32585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:38.495576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e54b-6791-4723-b771-bf1dcdf32585 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:38.495859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e54b-6791-4723-b771-bf1dcdf32585 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:38.496134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e54b-6791-4723-b771-bf1dcdf32585 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:38.573782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635e54b-6791-4723-b771-bf1dcdf32585 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:38.575349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635e54b-6791-4723-b771-bf1dcdf32585 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:38.636543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:38.689458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 210/427] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:38.696399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e54b-6791-4723-b771-bf1dcdf32585 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:38.800517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c8b32-f6b0-424a-a8b0-c6175c8b8597 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:38.823328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e54b-6791-4723-b771-bf1dcdf32585 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:38.867863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb0685-acdc-4c32-9ba2-395e17ea5b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:38.868418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb0685-acdc-4c32-9ba2-395e17ea5b09 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:38.868742 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb0685-acdc-4c32-9ba2-395e17ea5b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:38.869018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb0685-acdc-4c32-9ba2-395e17ea5b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:38.869752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb0685-acdc-4c32-9ba2-395e17ea5b09 None admin] RBAC: Adding query filter params (group.id=f52913c455004781af3e4524f4dbce9d, role.id=4f10622fcf4c4f708cd6b652d886ba3e, scope.project.id=6d01d0d61d804d4393d51b2d191f652d) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:38.870060 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb0685-acdc-4c32-9ba2-395e17ea5b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:38.871530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb0685-acdc-4c32-9ba2-395e17ea5b09 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:38.879351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 218/428] 10.222.0.85 () {56 vars in 1376 bytes} [Mon Aug 29 20:05:38 2022] GET /identity/v3/role_assignments?group.id=f52913c455004781af3e4524f4dbce9d&scope.project.id=6d01d0d61d804d4393d51b2d191f652d&role.id=4f10622fcf4c4f708cd6b652d886ba3e => generated 611 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:38.888403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456a9c5-f1fb-43fa-aebd-0e10f294496e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:38.889202 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456a9c5-f1fb-43fa-aebd-0e10f294496e None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:38.889585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456a9c5-f1fb-43fa-aebd-0e10f294496e None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:38.889960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456a9c5-f1fb-43fa-aebd-0e10f294496e None admin] PATH_INFO: `/v3/projects/6d01d0d61d804d4393d51b2d191f652d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:38.891235 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456a9c5-f1fb-43fa-aebd-0e10f294496e None admin] RBAC: Authorizing `identity:get_project(project_id=6d01d0d61d804d4393d51b2d191f652d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:38.893451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456a9c5-f1fb-43fa-aebd-0e10f294496e None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:38.895645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 211/429] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:38 2022] GET /identity/v3/projects/6d01d0d61d804d4393d51b2d191f652d => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:38.905440 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb0685-acdc-4c32-9ba2-395e17ea5b09 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:38.915887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:38.979765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6252fb2-1e4f-4f9b-a3e8-e306ffcaf023 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:38.980302 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6252fb2-1e4f-4f9b-a3e8-e306ffcaf023 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:38.980375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6252fb2-1e4f-4f9b-a3e8-e306ffcaf023 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:38.981775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 212/430] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:38.987324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61d20f-e85a-455e-800e-c8144dc469aa None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:38.987604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61d20f-e85a-455e-800e-c8144dc469aa None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:38.987862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61d20f-e85a-455e-800e-c8144dc469aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:39.008870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-2d61d20f-e85a-455e-800e-c8144dc469aa None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:39.010100 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2d61d20f-e85a-455e-800e-c8144dc469aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:39.049886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:39.081182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb0685-acdc-4c32-9ba2-395e17ea5b09 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:39.088877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 213/431] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:39.128225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6701257-24ba-47e2-8226-11cb46b2b28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:39.129087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6701257-24ba-47e2-8226-11cb46b2b28f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:39.129531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6701257-24ba-47e2-8226-11cb46b2b28f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:39.129948 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6701257-24ba-47e2-8226-11cb46b2b28f None admin] PATH_INFO: `/v3/groups/f52913c455004781af3e4524f4dbce9d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:39.131218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6701257-24ba-47e2-8226-11cb46b2b28f None admin] RBAC: Authorizing `identity:get_group(group_id=f52913c455004781af3e4524f4dbce9d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:39.132896 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6701257-24ba-47e2-8226-11cb46b2b28f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:39.134888 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 219/432] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:38 2022] GET /identity/v3/groups/f52913c455004781af3e4524f4dbce9d => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:39.140601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61d20f-e85a-455e-800e-c8144dc469aa None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:39.152778 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fa94a-8402-404c-a844-8647cb2f0009 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:39.153315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fa94a-8402-404c-a844-8647cb2f0009 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:39.153588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fa94a-8402-404c-a844-8647cb2f0009 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:39.155357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 220/433] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:05:39.160924 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63760b3-d6e4-454a-ae91-2cf53fd1603f None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:39.161442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63760b3-d6e4-454a-ae91-2cf53fd1603f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:39.161919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63760b3-d6e4-454a-ae91-2cf53fd1603f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:39.184036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-c63760b3-d6e4-454a-ae91-2cf53fd1603f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:39.185509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c63760b3-d6e4-454a-ae91-2cf53fd1603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:39.273883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 221/434] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:05:39.280210 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61d20f-e85a-455e-800e-c8144dc469aa None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:39.311075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1c4d9-4787-40ed-ad08-fa9bfe15af0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:39.311683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c4d9-4787-40ed-ad08-fa9bfe15af0c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:39.312008 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c4d9-4787-40ed-ad08-fa9bfe15af0c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:39.312311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c4d9-4787-40ed-ad08-fa9bfe15af0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:39.313037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c4d9-4787-40ed-ad08-fa9bfe15af0c None admin] RBAC: Adding query filter params (group.id=f52913c455004781af3e4524f4dbce9d, role.id=f1d8af7d158e4e02a6e77492178db259, scope.project.id=6d01d0d61d804d4393d51b2d191f652d) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:39.313341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c4d9-4787-40ed-ad08-fa9bfe15af0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:39.315433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c4d9-4787-40ed-ad08-fa9bfe15af0c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:39.326044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 214/435] 10.222.0.85 () {56 vars in 1376 bytes} [Mon Aug 29 20:05:39 2022] GET /identity/v3/role_assignments?group.id=f52913c455004781af3e4524f4dbce9d&scope.project.id=6d01d0d61d804d4393d51b2d191f652d&role.id=f1d8af7d158e4e02a6e77492178db259 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:39.418710 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63760b3-d6e4-454a-ae91-2cf53fd1603f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:39.421995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:39.528091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63760b3-d6e4-454a-ae91-2cf53fd1603f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:39.570480 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ef140-22dd-4eca-a4c5-662ca74dbf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:39.571081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1c4d9-4787-40ed-ad08-fa9bfe15af0c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:39.571219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef140-22dd-4eca-a4c5-662ca74dbf36 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:39.571432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef140-22dd-4eca-a4c5-662ca74dbf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:39.571751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef140-22dd-4eca-a4c5-662ca74dbf36 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:39.588627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:39.588821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ef140-22dd-4eca-a4c5-662ca74dbf36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-4d9ef140-22dd-4eca-a4c5-662ca74dbf36 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:39.594223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:39.600538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:39.601621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:39.601621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:39.601621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:39.601621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:39.601621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:39.601621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:39.601621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:39.601621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:39.601621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:39.601621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:39.601621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:39.601621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:39.601621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 222/436] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:05:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:39.606705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ef140-22dd-4eca-a4c5-662ca74dbf36 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:39.687906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:39.724428 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1c4d9-4787-40ed-ad08-fa9bfe15af0c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:39.742000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ef140-22dd-4eca-a4c5-662ca74dbf36 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:39.769512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40db86f-c578-4b17-b48c-44f9aa7e3712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:39.770248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40db86f-c578-4b17-b48c-44f9aa7e3712 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:39.770726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40db86f-c578-4b17-b48c-44f9aa7e3712 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:39.770988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40db86f-c578-4b17-b48c-44f9aa7e3712 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:39.778985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a1ea0-99e4-4818-81d7-6970afe19253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:39.779567 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a1ea0-99e4-4818-81d7-6970afe19253 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:39.779864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a1ea0-99e4-4818-81d7-6970afe19253 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:39.780366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a1ea0-99e4-4818-81d7-6970afe19253 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:39.781143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a1ea0-99e4-4818-81d7-6970afe19253 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:39.781443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a1ea0-99e4-4818-81d7-6970afe19253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:39.783338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a1ea0-99e4-4818-81d7-6970afe19253 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:39.784375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40db86f-c578-4b17-b48c-44f9aa7e3712 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:39.792858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 223/437] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-e40db86f-c578-4b17-b48c-44f9aa7e3712 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:39.793482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:39.794357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:39.795133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:39.795133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:39.795133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:39.795133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:39.795133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:39.795133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:39.795133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:39.795133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:39.795133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:39.795133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:39.795133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:05:39.795133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:39.795133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 215/438] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:39.798700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a1ea0-99e4-4818-81d7-6970afe19253 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:39.800975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40db86f-c578-4b17-b48c-44f9aa7e3712 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:39.884931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:39.914901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a1ea0-99e4-4818-81d7-6970afe19253 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:39.916348 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40db86f-c578-4b17-b48c-44f9aa7e3712 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:39.940803 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:39.952677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff30aea-0330-4365-b13b-40ce7462abd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:39.953279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff30aea-0330-4365-b13b-40ce7462abd7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:39.953563 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff30aea-0330-4365-b13b-40ce7462abd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:39.953658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311655a-0dae-4324-9079-47dc67a78b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:39.953837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff30aea-0330-4365-b13b-40ce7462abd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:39.954208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311655a-0dae-4324-9079-47dc67a78b93 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:39.954483 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311655a-0dae-4324-9079-47dc67a78b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:39.954757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311655a-0dae-4324-9079-47dc67a78b93 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:39.955416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311655a-0dae-4324-9079-47dc67a78b93 None admin] RBAC: Adding query filter params (name=member) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:39.955715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311655a-0dae-4324-9079-47dc67a78b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:39.957224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311655a-0dae-4324-9079-47dc67a78b93 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:39.965996 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 216/439] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:05:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:39.980949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7833a2-6c70-4b1c-909c-246316596300 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:39.980949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7833a2-6c70-4b1c-909c-246316596300 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:39.980949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7833a2-6c70-4b1c-909c-246316596300 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:39.980949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 217/440] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:39.980949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7833a2-6c70-4b1c-909c-246316596300 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:40.023618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff30aea-0330-4365-b13b-40ce7462abd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:40.025127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff30aea-0330-4365-b13b-40ce7462abd7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:40.075469 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7833a2-6c70-4b1c-909c-246316596300 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:40.109280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f83b4-6317-434a-b9bb-4171d0397cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:40.109879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f83b4-6317-434a-b9bb-4171d0397cfc None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.110153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f83b4-6317-434a-b9bb-4171d0397cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.111004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f83b4-6317-434a-b9bb-4171d0397cfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:40.113605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:40.125193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 224/441] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:05:40.126459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de233eb6-0b07-467b-9d4e-a990b0f82f92 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de233eb6-0b07-467b-9d4e-a990b0f82f92 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.127058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de233eb6-0b07-467b-9d4e-a990b0f82f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:40.144799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-de233eb6-0b07-467b-9d4e-a990b0f82f92 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:40.146060 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-de233eb6-0b07-467b-9d4e-a990b0f82f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:40.163536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:40.179708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f83b4-6317-434a-b9bb-4171d0397cfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:40.181193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f83b4-6317-434a-b9bb-4171d0397cfc None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:40.210983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 225/442] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:40.211754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de233eb6-0b07-467b-9d4e-a990b0f82f92 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:40.291069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 218/443] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:05:40.296752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f83b4-6317-434a-b9bb-4171d0397cfc None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:40.320721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de233eb6-0b07-467b-9d4e-a990b0f82f92 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:40.331419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:40.356428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea3a84-1055-4d36-b34a-5c4ea2d945d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:40.357054 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea3a84-1055-4d36-b34a-5c4ea2d945d1 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.357332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea3a84-1055-4d36-b34a-5c4ea2d945d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.357606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea3a84-1055-4d36-b34a-5c4ea2d945d1 None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:40.358722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea3a84-1055-4d36-b34a-5c4ea2d945d1 None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:40.360695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea3a84-1055-4d36-b34a-5c4ea2d945d1 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:40.366792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 226/444] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:40 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:40.367732 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea3a84-1055-4d36-b34a-5c4ea2d945d1 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:40.392059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:40.405950 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f83b4-6317-434a-b9bb-4171d0397cfc None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:40.444372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bb932-b785-454f-83da-1daa1e10705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:40.445004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bb932-b785-454f-83da-1daa1e10705e None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.445286 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bb932-b785-454f-83da-1daa1e10705e None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.445553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bb932-b785-454f-83da-1daa1e10705e None admin] PATH_INFO: `/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:40.446705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6bb932-b785-454f-83da-1daa1e10705e None admin] RBAC: Authorizing `identity:get_project(project_id=d120d625e6a84d1c9aa9d1bfab52dd24)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:40.448177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6bb932-b785-454f-83da-1daa1e10705e None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:40.449853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 219/445] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:40 2022] GET /identity/v3/projects/d120d625e6a84d1c9aa9d1bfab52dd24 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:40.455588 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bb932-b785-454f-83da-1daa1e10705e None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:40.475851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea3a84-1055-4d36-b34a-5c4ea2d945d1 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:40.509724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c8be2-6252-4c17-9b92-c2849efdb4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:40.510347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c8be2-6252-4c17-9b92-c2849efdb4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.510624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c8be2-6252-4c17-9b92-c2849efdb4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.510898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c8be2-6252-4c17-9b92-c2849efdb4d2 None admin] PATH_INFO: `/v3/groups/f52913c455004781af3e4524f4dbce9d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:40.512057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c8be2-6252-4c17-9b92-c2849efdb4d2 None admin] RBAC: Authorizing `identity:get_group(group_id=f52913c455004781af3e4524f4dbce9d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:40.513516 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c8be2-6252-4c17-9b92-c2849efdb4d2 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:40.515232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 227/446] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:40 2022] GET /identity/v3/groups/f52913c455004781af3e4524f4dbce9d => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:40.516071 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c8be2-6252-4c17-9b92-c2849efdb4d2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:40.557982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bb932-b785-454f-83da-1daa1e10705e None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:40.559542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:40.593276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0158f-f4f5-4203-84bc-4a764a4424d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:40.593918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0158f-f4f5-4203-84bc-4a764a4424d1 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.594184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0158f-f4f5-4203-84bc-4a764a4424d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.594456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0158f-f4f5-4203-84bc-4a764a4424d1 None admin] PATH_INFO: `/v3/groups/f52913c455004781af3e4524f4dbce9d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:40.595562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0158f-f4f5-4203-84bc-4a764a4424d1 None admin] RBAC: Authorizing `identity:get_group(group_id=f52913c455004781af3e4524f4dbce9d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:40.597045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0158f-f4f5-4203-84bc-4a764a4424d1 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:40.598619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 220/447] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:40 2022] GET /identity/v3/groups/f52913c455004781af3e4524f4dbce9d => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:40.599383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0158f-f4f5-4203-84bc-4a764a4424d1 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:40.607950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:40.617955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c8be2-6252-4c17-9b92-c2849efdb4d2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:40.653045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8acca-28f4-4197-acaf-1a3e6ab44f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:40.653673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8acca-28f4-4197-acaf-1a3e6ab44f0c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.653952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8acca-28f4-4197-acaf-1a3e6ab44f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.654248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8acca-28f4-4197-acaf-1a3e6ab44f0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:40.664450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8acca-28f4-4197-acaf-1a3e6ab44f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-61f8acca-28f4-4197-acaf-1a3e6ab44f0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:40.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:40.670367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 228/448] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:05:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:40.671371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8acca-28f4-4197-acaf-1a3e6ab44f0c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:40.706548 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0158f-f4f5-4203-84bc-4a764a4424d1 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:40.739319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86da73-6c07-42aa-9997-725b732fcd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:40.739840 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86da73-6c07-42aa-9997-725b732fcd23 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.740121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86da73-6c07-42aa-9997-725b732fcd23 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.740422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86da73-6c07-42aa-9997-725b732fcd23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:40.741142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86da73-6c07-42aa-9997-725b732fcd23 None admin] RBAC: Adding query filter params (group.id=f52913c455004781af3e4524f4dbce9d, role.id=f1d8af7d158e4e02a6e77492178db259, scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:40.741449 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86da73-6c07-42aa-9997-725b732fcd23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:40.742928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86da73-6c07-42aa-9997-725b732fcd23 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:40.749834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 221/449] 10.222.0.85 () {56 vars in 1376 bytes} [Mon Aug 29 20:05:40 2022] GET /identity/v3/role_assignments?group.id=f52913c455004781af3e4524f4dbce9d&scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24&role.id=f1d8af7d158e4e02a6e77492178db259 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:40.750007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86da73-6c07-42aa-9997-725b732fcd23 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:40.775872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8acca-28f4-4197-acaf-1a3e6ab44f0c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:40.790036 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:40.816277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8d52c-d19c-4ec3-8943-88277a03e295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:40.817004 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8d52c-d19c-4ec3-8943-88277a03e295 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.817357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8d52c-d19c-4ec3-8943-88277a03e295 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.817704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8d52c-d19c-4ec3-8943-88277a03e295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:40.818528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8d52c-d19c-4ec3-8943-88277a03e295 None admin] RBAC: Adding query filter params (group.id=f52913c455004781af3e4524f4dbce9d, role.id=4f10622fcf4c4f708cd6b652d886ba3e, scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:40.818923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8d52c-d19c-4ec3-8943-88277a03e295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:40.820551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8d52c-d19c-4ec3-8943-88277a03e295 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:40.827934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 229/450] 10.222.0.85 () {56 vars in 1376 bytes} [Mon Aug 29 20:05:40 2022] GET /identity/v3/role_assignments?group.id=f52913c455004781af3e4524f4dbce9d&scope.project.id=d120d625e6a84d1c9aa9d1bfab52dd24&role.id=4f10622fcf4c4f708cd6b652d886ba3e => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:40.877321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86da73-6c07-42aa-9997-725b732fcd23 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:40.918006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886292ea-182a-49c1-94cd-d785174c92e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:40.918632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886292ea-182a-49c1-94cd-d785174c92e4 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:40.918950 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886292ea-182a-49c1-94cd-d785174c92e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:40.919226 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886292ea-182a-49c1-94cd-d785174c92e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:40.919907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886292ea-182a-49c1-94cd-d785174c92e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:40.920243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886292ea-182a-49c1-94cd-d785174c92e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:40.921693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886292ea-182a-49c1-94cd-d785174c92e4 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:40.932015 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 222/451] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:40.937664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8d52c-d19c-4ec3-8943-88277a03e295 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:40.959602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:41.050884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8d52c-d19c-4ec3-8943-88277a03e295 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:41.085916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d9120-9aa5-40d4-93aa-6a26f819a6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:41.086552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d9120-9aa5-40d4-93aa-6a26f819a6af None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:41.086835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d9120-9aa5-40d4-93aa-6a26f819a6af None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:41.087107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d9120-9aa5-40d4-93aa-6a26f819a6af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:41.155642 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d9120-9aa5-40d4-93aa-6a26f819a6af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:41.157145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d9120-9aa5-40d4-93aa-6a26f819a6af None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:41.171759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:41.259239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 230/452] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:41.265270 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886292ea-182a-49c1-94cd-d785174c92e4 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:41.362555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:41.368014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886292ea-182a-49c1-94cd-d785174c92e4 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:41.399869 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026b4e4-d764-4a4b-a64a-3662c08471ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:41.400778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026b4e4-d764-4a4b-a64a-3662c08471ac None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:41.401057 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026b4e4-d764-4a4b-a64a-3662c08471ac None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:41.401330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026b4e4-d764-4a4b-a64a-3662c08471ac None admin] PATH_INFO: `/v3/projects/e08700a8f1f34f3488594693a51c7bb4` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:41.402431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026b4e4-d764-4a4b-a64a-3662c08471ac None admin] RBAC: Authorizing `identity:get_project(project_id=e08700a8f1f34f3488594693a51c7bb4)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:41.403862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026b4e4-d764-4a4b-a64a-3662c08471ac None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:41.405523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 223/453] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:05:41 2022] GET /identity/v3/projects/e08700a8f1f34f3488594693a51c7bb4 => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:41.411690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d9120-9aa5-40d4-93aa-6a26f819a6af None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:41.507174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d9120-9aa5-40d4-93aa-6a26f819a6af None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:41.520770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:41.540629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fce95a-27bd-461b-b766-65025ef55bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:41.541245 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fce95a-27bd-461b-b766-65025ef55bba None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:41.541525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fce95a-27bd-461b-b766-65025ef55bba None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:41.541794 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fce95a-27bd-461b-b766-65025ef55bba None admin] PATH_INFO: `/v3/groups/67e9612ed98343fabf0bc5f9c55fc83f` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:41.542882 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fce95a-27bd-461b-b766-65025ef55bba None admin] RBAC: Authorizing `identity:get_group(group_id=67e9612ed98343fabf0bc5f9c55fc83f)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:41.544378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fce95a-27bd-461b-b766-65025ef55bba None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:41.545983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 231/454] 10.222.0.85 () {56 vars in 1180 bytes} [Mon Aug 29 20:05:41 2022] GET /identity/v3/groups/67e9612ed98343fabf0bc5f9c55fc83f => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:41.551919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026b4e4-d764-4a4b-a64a-3662c08471ac None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:41.651246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026b4e4-d764-4a4b-a64a-3662c08471ac None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:41.686814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ef9db-b76b-42be-a964-72d6b35e5e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:41.687412 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ef9db-b76b-42be-a964-72d6b35e5e6b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:41.687695 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ef9db-b76b-42be-a964-72d6b35e5e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:41.687968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ef9db-b76b-42be-a964-72d6b35e5e6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:41.689360 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ef9db-b76b-42be-a964-72d6b35e5e6b None admin] RBAC: Adding query filter params (group.id=67e9612ed98343fabf0bc5f9c55fc83f, role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=e08700a8f1f34f3488594693a51c7bb4) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:41.689667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ef9db-b76b-42be-a964-72d6b35e5e6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:41.691098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ef9db-b76b-42be-a964-72d6b35e5e6b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:41.696788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 224/455] 10.222.0.85 () {56 vars in 1376 bytes} [Mon Aug 29 20:05:41 2022] GET /identity/v3/role_assignments?group.id=67e9612ed98343fabf0bc5f9c55fc83f&scope.project.id=e08700a8f1f34f3488594693a51c7bb4&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:44.086999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45334200-7e00-4c3e-b5fa-0c72743c9390 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:44.087553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45334200-7e00-4c3e-b5fa-0c72743c9390 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:44.087553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45334200-7e00-4c3e-b5fa-0c72743c9390 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:44.089152 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 232/456] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:44.105740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af913370-af55-4d77-9ca1-7e19268574d6 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:44.106661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af913370-af55-4d77-9ca1-7e19268574d6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:44.107128 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af913370-af55-4d77-9ca1-7e19268574d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:44.123204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77658859-e460-43fa-ab18-0d41aab8330e None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:44.123857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77658859-e460-43fa-ab18-0d41aab8330e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:44.124440 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77658859-e460-43fa-ab18-0d41aab8330e None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:44.126302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 233/457] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:44.131958 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16235c31-8d81-4977-9a03-2b2939cbbe6f None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:44.132493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16235c31-8d81-4977-9a03-2b2939cbbe6f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:44.134255 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16235c31-8d81-4977-9a03-2b2939cbbe6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:44.137285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-af913370-af55-4d77-9ca1-7e19268574d6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:44.138951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-af913370-af55-4d77-9ca1-7e19268574d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:44.166054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:44.167695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-16235c31-8d81-4977-9a03-2b2939cbbe6f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:44.169200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-16235c31-8d81-4977-9a03-2b2939cbbe6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:44.254466 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 225/458] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:44.293823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 234/459] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:05:44.486975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc8017-537d-40c4-8059-090b102ce950 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:44.487619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc8017-537d-40c4-8059-090b102ce950 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:44.488121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc8017-537d-40c4-8059-090b102ce950 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:44.490102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 226/460] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:44.495165 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cd89d-2be0-49b5-8545-5b6310a692e3 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:44.495682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cd89d-2be0-49b5-8545-5b6310a692e3 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:44.496157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cd89d-2be0-49b5-8545-5b6310a692e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:44.525170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-cb8cd89d-2be0-49b5-8545-5b6310a692e3 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:44.526944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-cb8cd89d-2be0-49b5-8545-5b6310a692e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:44.540266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc8017-537d-40c4-8059-090b102ce950 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:44.639994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 235/461] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:44.723823 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc8017-537d-40c4-8059-090b102ce950 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:44.776997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9f449-7a6e-45df-92c3-0f72a9970504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:44.777969 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9f449-7a6e-45df-92c3-0f72a9970504 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:44.778452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9f449-7a6e-45df-92c3-0f72a9970504 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:44.778924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9f449-7a6e-45df-92c3-0f72a9970504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:44.790160 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9f449-7a6e-45df-92c3-0f72a9970504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:44.791907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9f449-7a6e-45df-92c3-0f72a9970504 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-25b9f449-7a6e-45df-92c3-0f72a9970504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:44.815691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:44.816694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:44.817563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:44.817563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:44.817563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:44.819657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 227/462] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:44.825064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cd89d-2be0-49b5-8545-5b6310a692e3 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:44.843346 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9f449-7a6e-45df-92c3-0f72a9970504 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:44.860791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:44.885643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:44.951789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cd89d-2be0-49b5-8545-5b6310a692e3 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:44.961334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9f449-7a6e-45df-92c3-0f72a9970504 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:44.989042 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758baeb-197d-4b1f-8c1d-3ca96ba10890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:44.989878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758baeb-197d-4b1f-8c1d-3ca96ba10890 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:44.990377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758baeb-197d-4b1f-8c1d-3ca96ba10890 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:44.990880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758baeb-197d-4b1f-8c1d-3ca96ba10890 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:44.991843 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758baeb-197d-4b1f-8c1d-3ca96ba10890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:44.992328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758baeb-197d-4b1f-8c1d-3ca96ba10890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:44.993690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758baeb-197d-4b1f-8c1d-3ca96ba10890 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:44.996249 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f414dd4-f792-4d74-bc16-213d37f123c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:44.996880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f414dd4-f792-4d74-bc16-213d37f123c0 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:44.997159 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f414dd4-f792-4d74-bc16-213d37f123c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:44.997430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f414dd4-f792-4d74-bc16-213d37f123c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:45.006436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 236/463] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:45.007996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9aa4d-463c-40a0-971d-9295c1c44062 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:45.008309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9aa4d-463c-40a0-971d-9295c1c44062 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:45.008584 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9aa4d-463c-40a0-971d-9295c1c44062 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:45.012553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 237/464] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:45.012553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9aa4d-463c-40a0-971d-9295c1c44062 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:45.014933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f414dd4-f792-4d74-bc16-213d37f123c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:45.016497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f414dd4-f792-4d74-bc16-213d37f123c0 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-1f414dd4-f792-4d74-bc16-213d37f123c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:45.031433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:45.032979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:45.038879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:45.038879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:45.038879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:45.038879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 228/465] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:45.038879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f414dd4-f792-4d74-bc16-213d37f123c0 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:45.099515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:45.125196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9aa4d-463c-40a0-971d-9295c1c44062 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:45.127187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:45.146426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f414dd4-f792-4d74-bc16-213d37f123c0 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:45.161987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4667ff4-49ae-4c33-bdbe-522f66af033a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:45.162636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4667ff4-49ae-4c33-bdbe-522f66af033a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:45.163125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4667ff4-49ae-4c33-bdbe-522f66af033a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:45.163557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4667ff4-49ae-4c33-bdbe-522f66af033a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:45.174930 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4667ff4-49ae-4c33-bdbe-522f66af033a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:45.176506 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4667ff4-49ae-4c33-bdbe-522f66af033a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:45.186926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93ff20-93ba-4ab5-b02e-7f0a8445293d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:45.187550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93ff20-93ba-4ab5-b02e-7f0a8445293d None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:45.187984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93ff20-93ba-4ab5-b02e-7f0a8445293d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:45.188350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93ff20-93ba-4ab5-b02e-7f0a8445293d None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:45.188975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93ff20-93ba-4ab5-b02e-7f0a8445293d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:45.190430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93ff20-93ba-4ab5-b02e-7f0a8445293d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-f4667ff4-49ae-4c33-bdbe-522f66af033a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:45.193203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:45.194047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:45.200022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:45.200022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:45.200022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:45.200022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 238/466] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:45.200022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cdf0c-dcfb-4e7c-ab3e-cec3a9285c50 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:45.200022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cdf0c-dcfb-4e7c-ab3e-cec3a9285c50 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:45.200022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cdf0c-dcfb-4e7c-ab3e-cec3a9285c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:45.216808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-809cdf0c-dcfb-4e7c-ab3e-cec3a9285c50 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:45.217995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-809cdf0c-dcfb-4e7c-ab3e-cec3a9285c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:45.284981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 239/467] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:45.285592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cdf0c-dcfb-4e7c-ab3e-cec3a9285c50 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:45.340792 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 229/468] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:05:45 2022] POST /identity/v3/users => generated 246 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:05:45.343354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93ff20-93ba-4ab5-b02e-7f0a8445293d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:45.432626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:45.468505 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cdf0c-dcfb-4e7c-ab3e-cec3a9285c50 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:45.526140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa466c-8139-4d86-8e97-a0b0089d0e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:45.527131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa466c-8139-4d86-8e97-a0b0089d0e2a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:45.527611 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa466c-8139-4d86-8e97-a0b0089d0e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:45.528085 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa466c-8139-4d86-8e97-a0b0089d0e2a None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:45.528954 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa466c-8139-4d86-8e97-a0b0089d0e2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:45.529474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa466c-8139-4d86-8e97-a0b0089d0e2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:45.531112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa466c-8139-4d86-8e97-a0b0089d0e2a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:45.534168 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93ff20-93ba-4ab5-b02e-7f0a8445293d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:45.544891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 240/469] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:45.551449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa466c-8139-4d86-8e97-a0b0089d0e2a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:45.558264 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:45.577273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb129a-b982-4040-a642-9867bc66ec5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:45.577662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb129a-b982-4040-a642-9867bc66ec5c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:45.577951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb129a-b982-4040-a642-9867bc66ec5c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:45.578227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb129a-b982-4040-a642-9867bc66ec5c None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:45.579404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb129a-b982-4040-a642-9867bc66ec5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:45.579808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb129a-b982-4040-a642-9867bc66ec5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:45.581125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb129a-b982-4040-a642-9867bc66ec5c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:45.600872 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 230/470] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:45.602064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb129a-b982-4040-a642-9867bc66ec5c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:45.683064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa466c-8139-4d86-8e97-a0b0089d0e2a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:45.715960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb129a-b982-4040-a642-9867bc66ec5c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:45.720118 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa414a-fffa-4d91-9c41-f2d84551002d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:45.720765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa414a-fffa-4d91-9c41-f2d84551002d None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:45.721047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa414a-fffa-4d91-9c41-f2d84551002d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:45.721323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa414a-fffa-4d91-9c41-f2d84551002d None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:45.721973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa414a-fffa-4d91-9c41-f2d84551002d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:45.723425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa414a-fffa-4d91-9c41-f2d84551002d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:45.756185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fa5e0-7b84-4fc0-b1c8-eab6cf6fadd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:45.756875 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fa5e0-7b84-4fc0-b1c8-eab6cf6fadd6 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:45.757152 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fa5e0-7b84-4fc0-b1c8-eab6cf6fadd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:45.757424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fa5e0-7b84-4fc0-b1c8-eab6cf6fadd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:45.768606 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fa5e0-7b84-4fc0-b1c8-eab6cf6fadd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:45.770058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fa5e0-7b84-4fc0-b1c8-eab6cf6fadd6 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-624fa5e0-7b84-4fc0-b1c8-eab6cf6fadd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:45.785341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:45.786327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:45.787184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:45.787184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:45.787184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:45.787184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 231/471] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:45.787697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fa5e0-7b84-4fc0-b1c8-eab6cf6fadd6 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:45.818898 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:45.885142 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:45.886036 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fa5e0-7b84-4fc0-b1c8-eab6cf6fadd6 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:45.896357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 241/472] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:05:45 2022] POST /identity/v3/users => generated 248 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:05:45.896357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c883848-c76d-4a67-8033-ea3d08275af2 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:45.896357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c883848-c76d-4a67-8033-ea3d08275af2 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:45.896357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c883848-c76d-4a67-8033-ea3d08275af2 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:45.896357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 242/473] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:45.896357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c883848-c76d-4a67-8033-ea3d08275af2 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:45.932278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b49fa-dd5a-4fd9-a814-92fcf5fc020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:45.932905 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b49fa-dd5a-4fd9-a814-92fcf5fc020c None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:45.933186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b49fa-dd5a-4fd9-a814-92fcf5fc020c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:45.933456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b49fa-dd5a-4fd9-a814-92fcf5fc020c None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:45.934221 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b49fa-dd5a-4fd9-a814-92fcf5fc020c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:45.935724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b49fa-dd5a-4fd9-a814-92fcf5fc020c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:46.027358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c883848-c76d-4a67-8033-ea3d08275af2 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:46.063359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31418f6c-9ed4-4726-a03a-a9c725fb5a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:46.064359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31418f6c-9ed4-4726-a03a-a9c725fb5a76 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:46.065926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31418f6c-9ed4-4726-a03a-a9c725fb5a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:46.066370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31418f6c-9ed4-4726-a03a-a9c725fb5a76 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:46.067161 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31418f6c-9ed4-4726-a03a-a9c725fb5a76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:46.067615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31418f6c-9ed4-4726-a03a-a9c725fb5a76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:46.069247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31418f6c-9ed4-4726-a03a-a9c725fb5a76 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:46.082050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 243/474] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:46.083485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e76e2-6407-489e-a81a-70e878b99dda None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:46.083772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e76e2-6407-489e-a81a-70e878b99dda None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:46.084039 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e76e2-6407-489e-a81a-70e878b99dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:46.103462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-440e76e2-6407-489e-a81a-70e878b99dda None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:46.104676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-440e76e2-6407-489e-a81a-70e878b99dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:46.168367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:46.183753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 244/475] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:46.183753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e76e2-6407-489e-a81a-70e878b99dda None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:46.285264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e76e2-6407-489e-a81a-70e878b99dda None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:46.320129 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 232/476] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:05:45 2022] POST /identity/v3/users => generated 248 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:05:46.331077 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23dfa10-89f3-476a-ac6b-c586bba3a12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:46.331705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dfa10-89f3-476a-ac6b-c586bba3a12c None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:46.331985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dfa10-89f3-476a-ac6b-c586bba3a12c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:46.356345 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dfa10-89f3-476a-ac6b-c586bba3a12c None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:46.356345 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23dfa10-89f3-476a-ac6b-c586bba3a12c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:46.356345 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23dfa10-89f3-476a-ac6b-c586bba3a12c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:46.496691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b49fa-dd5a-4fd9-a814-92fcf5fc020c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:46.541355 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 245/477] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:05:46 2022] POST /identity/v3/users => generated 249 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:05:46.597300 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:46.619280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b49fa-dd5a-4fd9-a814-92fcf5fc020c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:46.663628 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931337f-7acb-4c3b-b865-381a03634de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:46.664264 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931337f-7acb-4c3b-b865-381a03634de2 None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:46.664544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931337f-7acb-4c3b-b865-381a03634de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:46.664815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931337f-7acb-4c3b-b865-381a03634de2 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:46.665499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931337f-7acb-4c3b-b865-381a03634de2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:46.666960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931337f-7acb-4c3b-b865-381a03634de2 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:46.729571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.sql.core [None req-9931337f-7acb-4c3b-b865-381a03634de2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 20:05:46.729571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 20:05:46.729571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [parameters: {'id': 'a0f32406da154f1e8a63d184ffb89cec', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 20:05:46.729571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81858) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-9931337f-7acb-4c3b-b865-381a03634de2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application result.read() Aug 29 20:05:46.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return ref.to_dict() Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:05:46.748257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application yield resource Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise self.value Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application session.commit() Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 20:05:46.749248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self.session.flush() Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application compat.raise_( Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 20:05:46.750099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:05:46.751072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application result.read() Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:05:46.753685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [parameters: {'id': 'a0f32406da154f1e8a63d184ffb89cec', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:46.756053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 20:05:46.758030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:46.761546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 233/478] 10.222.0.85 () {60 vars in 1131 bytes} [Mon Aug 29 20:05:46 2022] POST /identity/v3/roles => generated 144 bytes in 254 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:05:46.761695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23dfa10-89f3-476a-ac6b-c586bba3a12c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:46.817215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:46.874728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23dfa10-89f3-476a-ac6b-c586bba3a12c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:46.912858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dc1c1-2c16-4d56-8954-c9367395bd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:46.913826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dc1c1-2c16-4d56-8954-c9367395bd74 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:46.914288 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dc1c1-2c16-4d56-8954-c9367395bd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:46.914763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dc1c1-2c16-4d56-8954-c9367395bd74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:46.926212 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799dc1c1-2c16-4d56-8954-c9367395bd74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-799dc1c1-2c16-4d56-8954-c9367395bd74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:46.932104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:46.935396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:46.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:46.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:46.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:46.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:46.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:46.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:46.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:46.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:46.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:46.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:46.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:05:46.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:46.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 246/479] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:05:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:46.939339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931337f-7acb-4c3b-b865-381a03634de2 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:47.055973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931337f-7acb-4c3b-b865-381a03634de2 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:47.096791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d47ad-7aa5-455b-bb77-1d46e9d5f180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:47.097704 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d47ad-7aa5-455b-bb77-1d46e9d5f180 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:47.098166 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d47ad-7aa5-455b-bb77-1d46e9d5f180 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:47.098645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d47ad-7aa5-455b-bb77-1d46e9d5f180 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:47.099497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d47ad-7aa5-455b-bb77-1d46e9d5f180 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:47.100029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d47ad-7aa5-455b-bb77-1d46e9d5f180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:47.101676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d47ad-7aa5-455b-bb77-1d46e9d5f180 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:47.111454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 234/480] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:47.538707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498a1a8-719d-4c65-b25d-4d663c3f7e21 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:47.539677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498a1a8-719d-4c65-b25d-4d663c3f7e21 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:47.539963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498a1a8-719d-4c65-b25d-4d663c3f7e21 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:47.541516 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 247/481] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:47.546602 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed4513-c4ca-4c65-994b-aa8bc928a26e None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:47.547026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed4513-c4ca-4c65-994b-aa8bc928a26e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:47.547208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed4513-c4ca-4c65-994b-aa8bc928a26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:47.567596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-8eed4513-c4ca-4c65-994b-aa8bc928a26e None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:47.568816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8eed4513-c4ca-4c65-994b-aa8bc928a26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:47.576995 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:47.652220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 235/482] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:48.018560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498a1a8-719d-4c65-b25d-4d663c3f7e21 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:48.159561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498a1a8-719d-4c65-b25d-4d663c3f7e21 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:48.170558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:48.207402 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ba7a7-3711-4060-bec9-72f40ed9338f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:48.208459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ba7a7-3711-4060-bec9-72f40ed9338f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:48.208996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ba7a7-3711-4060-bec9-72f40ed9338f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:48.209504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ba7a7-3711-4060-bec9-72f40ed9338f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:48.225090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ba7a7-3711-4060-bec9-72f40ed9338f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-c78ba7a7-3711-4060-bec9-72f40ed9338f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:48.230174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:48.231137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:48.231935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:48.231935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:48.231935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:48.231935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:48.231935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:48.231935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:48.231935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:48.231935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:48.231935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:48.231935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:48.231935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:48.231935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:48.231935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 248/483] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:05:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:48.237260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed4513-c4ca-4c65-994b-aa8bc928a26e None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:48.239114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19e74a-0726-406b-b2bc-d1e070fa49e3 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:48.239534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19e74a-0726-406b-b2bc-d1e070fa49e3 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:48.239740 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19e74a-0726-406b-b2bc-d1e070fa49e3 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:48.241317 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 249/484] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:05:48.254718 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ab95e-c802-478d-ab09-0612261fed3c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:48.255039 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ab95e-c802-478d-ab09-0612261fed3c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:48.255330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ab95e-c802-478d-ab09-0612261fed3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:48.284611 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-068ab95e-c802-478d-ab09-0612261fed3c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:48.286459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-068ab95e-c802-478d-ab09-0612261fed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:48.414014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed4513-c4ca-4c65-994b-aa8bc928a26e None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:48.416692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 250/485] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:05:48.475255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a13ca-9a75-473f-989f-b880efad1ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:48.475875 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a13ca-9a75-473f-989f-b880efad1ccf None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:48.476158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a13ca-9a75-473f-989f-b880efad1ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:48.476456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a13ca-9a75-473f-989f-b880efad1ccf None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:48.477166 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a13ca-9a75-473f-989f-b880efad1ccf None admin] RBAC: Adding query filter params (name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:48.477480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a13ca-9a75-473f-989f-b880efad1ccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:48.478865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a13ca-9a75-473f-989f-b880efad1ccf None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:48.491863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 236/486] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:05:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:48.496952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ab95e-c802-478d-ab09-0612261fed3c None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:48.600618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:48.608494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ab95e-c802-478d-ab09-0612261fed3c None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:48.666419 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8341e-db63-4be2-a689-9de63911b9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:48.667492 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8341e-db63-4be2-a689-9de63911b9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:48.667973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8341e-db63-4be2-a689-9de63911b9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:48.668468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8341e-db63-4be2-a689-9de63911b9c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:48.686407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8341e-db63-4be2-a689-9de63911b9c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:48.688365 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8341e-db63-4be2-a689-9de63911b9c3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-17c8341e-db63-4be2-a689-9de63911b9c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:48.710866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:48.711817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:48.712714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:48.712714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:48.712714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:48.714788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 251/487] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:48.722119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a13ca-9a75-473f-989f-b880efad1ccf None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:48.737328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:48.768540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ca657-67e9-4065-855e-be07621cb37f None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:48.768882 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ca657-67e9-4065-855e-be07621cb37f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:48.769140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ca657-67e9-4065-855e-be07621cb37f None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:48.770751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 252/488] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:48.777406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae7e65-a8f8-4528-9da6-595f93a12c05 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:48.777815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae7e65-a8f8-4528-9da6-595f93a12c05 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:48.778092 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae7e65-a8f8-4528-9da6-595f93a12c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:48.799905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-d6ae7e65-a8f8-4528-9da6-595f93a12c05 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:48.801131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-d6ae7e65-a8f8-4528-9da6-595f93a12c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:48.881707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 253/489] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:48.882533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae7e65-a8f8-4528-9da6-595f93a12c05 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:48.893380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a13ca-9a75-473f-989f-b880efad1ccf None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:48.961032 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e6b7a-6d7a-4cf0-967a-313ff74036a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:48.962802 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e6b7a-6d7a-4cf0-967a-313ff74036a8 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:48.963312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e6b7a-6d7a-4cf0-967a-313ff74036a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:48.963788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e6b7a-6d7a-4cf0-967a-313ff74036a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:48.964805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e6b7a-6d7a-4cf0-967a-313ff74036a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:48.966693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e6b7a-6d7a-4cf0-967a-313ff74036a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:48.969119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e6b7a-6d7a-4cf0-967a-313ff74036a8 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:48.995543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 237/490] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:49.002328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e6b7a-6d7a-4cf0-967a-313ff74036a8 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:49.035050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:49.082616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae7e65-a8f8-4528-9da6-595f93a12c05 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:49.102599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:49.128099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a5f5d-4bcf-44be-8d6f-215a206fc305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:49.129140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a5f5d-4bcf-44be-8d6f-215a206fc305 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:49.129619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a5f5d-4bcf-44be-8d6f-215a206fc305 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:49.130047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a5f5d-4bcf-44be-8d6f-215a206fc305 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:49.140848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a5f5d-4bcf-44be-8d6f-215a206fc305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-462a5f5d-4bcf-44be-8d6f-215a206fc305 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:49.146119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:49.146999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:49.147770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:49.147770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:49.147770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:49.147770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:49.147770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:49.147770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:49.147770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:49.147770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:49.147770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:49.147770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:49.147770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:49.147770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:49.150025 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 254/491] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:05:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:49.151630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a225340-ca75-4b7c-aca9-aedad65bed05 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:49.152282 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a225340-ca75-4b7c-aca9-aedad65bed05 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:49.152750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a225340-ca75-4b7c-aca9-aedad65bed05 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:49.154477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 255/492] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:49.156028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e6b7a-6d7a-4cf0-967a-313ff74036a8 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:49.159704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a225340-ca75-4b7c-aca9-aedad65bed05 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:49.198604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c165d59-cb4b-4622-8ed2-e6a3383a874c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:49.199185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c165d59-cb4b-4622-8ed2-e6a3383a874c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:49.199470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c165d59-cb4b-4622-8ed2-e6a3383a874c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:49.199743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c165d59-cb4b-4622-8ed2-e6a3383a874c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:49.208535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c165d59-cb4b-4622-8ed2-e6a3383a874c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:49.209988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c165d59-cb4b-4622-8ed2-e6a3383a874c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-8c165d59-cb4b-4622-8ed2-e6a3383a874c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:05:49.224144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:49.225188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:49.226068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:49.226068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:49.226068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:05:49.226068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:05:49.226068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:05:49.226068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:05:49.226068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:05:49.226068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:05:49.226068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:05:49.226068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:49.226068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 238/493] 10.222.0.85 () {56 vars in 1093 bytes} [Mon Aug 29 20:05:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:05:49.226708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c73d4c-9360-4dae-a83b-2a9c26eb7314 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:49.227486 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c73d4c-9360-4dae-a83b-2a9c26eb7314 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:49.227486 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c73d4c-9360-4dae-a83b-2a9c26eb7314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:49.245759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-33c73d4c-9360-4dae-a83b-2a9c26eb7314 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:49.246945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-33c73d4c-9360-4dae-a83b-2a9c26eb7314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:49.288407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a225340-ca75-4b7c-aca9-aedad65bed05 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:49.320037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 239/494] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:49.321317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c73d4c-9360-4dae-a83b-2a9c26eb7314 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:49.334606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc23e0-95e8-4048-9771-b9e93e4073c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:49.335536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc23e0-95e8-4048-9771-b9e93e4073c5 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:49.336078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc23e0-95e8-4048-9771-b9e93e4073c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:49.336650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc23e0-95e8-4048-9771-b9e93e4073c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:49.337506 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc23e0-95e8-4048-9771-b9e93e4073c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:49.338092 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc23e0-95e8-4048-9771-b9e93e4073c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:49.339668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc23e0-95e8-4048-9771-b9e93e4073c5 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:49.346413 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:49.350604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 256/495] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:05:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:49.351793 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc23e0-95e8-4048-9771-b9e93e4073c5 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:49.398952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:49.465086 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c73d4c-9360-4dae-a83b-2a9c26eb7314 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:49.472654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc23e0-95e8-4048-9771-b9e93e4073c5 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:49.512630 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bba5e-ba36-40ab-9ed0-4dd6c5035fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:49.513640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bba5e-ba36-40ab-9ed0-4dd6c5035fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:49.514059 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bba5e-ba36-40ab-9ed0-4dd6c5035fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:49.514440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bba5e-ba36-40ab-9ed0-4dd6c5035fb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:49.517072 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860aa69-64d1-42a2-8767-c1b34532142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:49.518346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860aa69-64d1-42a2-8767-c1b34532142c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:49.518779 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860aa69-64d1-42a2-8767-c1b34532142c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:49.519220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860aa69-64d1-42a2-8767-c1b34532142c None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:49.520098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860aa69-64d1-42a2-8767-c1b34532142c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:49.520569 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860aa69-64d1-42a2-8767-c1b34532142c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:49.522694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860aa69-64d1-42a2-8767-c1b34532142c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:49.527579 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bba5e-ba36-40ab-9ed0-4dd6c5035fb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-310bba5e-ba36-40ab-9ed0-4dd6c5035fb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:49.533909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:49.535006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:49.536012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:49.536012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:49.536012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:49.536012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:49.536012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:49.536012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:49.536012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:49.536012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:49.536012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:49.536012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:49.536012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:49.536012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:49.542489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 240/496] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:05:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:49.543455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bba5e-ba36-40ab-9ed0-4dd6c5035fb0 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:49.574918 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 257/497] 10.222.0.85 () {56 vars in 1133 bytes} [Mon Aug 29 20:05:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:49.576811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860aa69-64d1-42a2-8767-c1b34532142c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:49.649363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:49.669624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bba5e-ba36-40ab-9ed0-4dd6c5035fb0 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:49.684740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:49.694636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860aa69-64d1-42a2-8767-c1b34532142c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:49.704745 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cd508-30f2-4989-bfd0-52dce4327854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:49.705371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cd508-30f2-4989-bfd0-52dce4327854 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:49.705648 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cd508-30f2-4989-bfd0-52dce4327854 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:49.705916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cd508-30f2-4989-bfd0-52dce4327854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:49.716133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cd508-30f2-4989-bfd0-52dce4327854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:49.717609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cd508-30f2-4989-bfd0-52dce4327854 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:49.731446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c495aa-443c-4c93-999c-db3122a5130f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:49.732057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c495aa-443c-4c93-999c-db3122a5130f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:49.732374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c495aa-443c-4c93-999c-db3122a5130f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:49.732652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c495aa-443c-4c93-999c-db3122a5130f None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:49.733323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c495aa-443c-4c93-999c-db3122a5130f None admin] RBAC: Adding query filter params (name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:49.733663 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c495aa-443c-4c93-999c-db3122a5130f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-a70cd508-30f2-4989-bfd0-52dce4327854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:49.734680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:49.736508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:49.737344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:49.737344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:49.737344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:49.737541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c495aa-443c-4c93-999c-db3122a5130f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:49.737620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 241/498] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:49.737620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cd508-30f2-4989-bfd0-52dce4327854 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:49.753499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 258/499] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:05:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:49.753499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c495aa-443c-4c93-999c-db3122a5130f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:49.846093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cd508-30f2-4989-bfd0-52dce4327854 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:49.854022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c495aa-443c-4c93-999c-db3122a5130f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:49.874060 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:49.879533 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b85fcc-160f-49f7-bf73-3b702164aeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:49.880141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b85fcc-160f-49f7-bf73-3b702164aeff None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:49.880452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b85fcc-160f-49f7-bf73-3b702164aeff None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:49.884187 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b85fcc-160f-49f7-bf73-3b702164aeff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:49.887616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82681b-2c35-44a1-9676-120e2db48e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:49.888275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82681b-2c35-44a1-9676-120e2db48e9b None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:49.888553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82681b-2c35-44a1-9676-120e2db48e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:49.888952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82681b-2c35-44a1-9676-120e2db48e9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:49.899840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82681b-2c35-44a1-9676-120e2db48e9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-0c82681b-2c35-44a1-9676-120e2db48e9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:49.905748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:49.906660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:49.908055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:49.908055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:49.908055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:49.908055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:49.908055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:49.908055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:49.908055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:49.908055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:49.908055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:49.908055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:49.908055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:49.908055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:49.908055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 259/500] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:05:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:49.908743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82681b-2c35-44a1-9676-120e2db48e9b None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:49.918385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:49.952731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b85fcc-160f-49f7-bf73-3b702164aeff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:49.954726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b85fcc-160f-49f7-bf73-3b702164aeff None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:50.008997 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82681b-2c35-44a1-9676-120e2db48e9b None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:50.043500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c095ca-e464-43a1-9bf4-494fc392c865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:50.044098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c095ca-e464-43a1-9bf4-494fc392c865 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.044403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c095ca-e464-43a1-9bf4-494fc392c865 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.044671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c095ca-e464-43a1-9bf4-494fc392c865 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.045297 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c095ca-e464-43a1-9bf4-494fc392c865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:50.045596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c095ca-e464-43a1-9bf4-494fc392c865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:50.046955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c095ca-e464-43a1-9bf4-494fc392c865 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:50.055666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 242/501] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:50.056546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b85fcc-160f-49f7-bf73-3b702164aeff None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:50.061420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 260/502] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:50.062625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a257a68-93f0-413f-bf60-3f70a8ed6ccc None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.062931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a257a68-93f0-413f-bf60-3f70a8ed6ccc None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.063885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a257a68-93f0-413f-bf60-3f70a8ed6ccc None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.068541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 261/503] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:50.068541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a257a68-93f0-413f-bf60-3f70a8ed6ccc None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:50.110785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:50.123516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:50.160061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b85fcc-160f-49f7-bf73-3b702164aeff None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:50.167360 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a257a68-93f0-413f-bf60-3f70a8ed6ccc None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:50.195001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83d6fb-b2dc-42ed-8154-123c7d898010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:50.195612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83d6fb-b2dc-42ed-8154-123c7d898010 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.195893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83d6fb-b2dc-42ed-8154-123c7d898010 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.196190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83d6fb-b2dc-42ed-8154-123c7d898010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.201846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1ba29-3575-439b-aaae-f24dea2fd577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:50.202438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1ba29-3575-439b-aaae-f24dea2fd577 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.202718 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1ba29-3575-439b-aaae-f24dea2fd577 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.202987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1ba29-3575-439b-aaae-f24dea2fd577 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.203695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1ba29-3575-439b-aaae-f24dea2fd577 None admin] RBAC: Adding query filter params (name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:50.203994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1ba29-3575-439b-aaae-f24dea2fd577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:50.205542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1ba29-3575-439b-aaae-f24dea2fd577 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:50.206455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83d6fb-b2dc-42ed-8154-123c7d898010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:50.207928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83d6fb-b2dc-42ed-8154-123c7d898010 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:50.213564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 262/504] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:05:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:50.214342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1ba29-3575-439b-aaae-f24dea2fd577 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-da83d6fb-b2dc-42ed-8154-123c7d898010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:50.227593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:50.228760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:50.229586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:50.229586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:50.229586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:50.229586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 243/505] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:50.229586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83d6fb-b2dc-42ed-8154-123c7d898010 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:50.315871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1ba29-3575-439b-aaae-f24dea2fd577 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:50.317488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:50.329109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83d6fb-b2dc-42ed-8154-123c7d898010 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:50.346431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:50.349118 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb7806-8ae6-4e1a-b463-8d53649487d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:50.349927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb7806-8ae6-4e1a-b463-8d53649487d9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.350212 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb7806-8ae6-4e1a-b463-8d53649487d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.350488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb7806-8ae6-4e1a-b463-8d53649487d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.361354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb7806-8ae6-4e1a-b463-8d53649487d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:50.362799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb7806-8ae6-4e1a-b463-8d53649487d9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:50.363598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90268ff8-66f4-45ac-8437-f91f76ee3285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:50.364287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90268ff8-66f4-45ac-8437-f91f76ee3285 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.364574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90268ff8-66f4-45ac-8437-f91f76ee3285 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.364851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90268ff8-66f4-45ac-8437-f91f76ee3285 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.372320 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90268ff8-66f4-45ac-8437-f91f76ee3285 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:50.373755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90268ff8-66f4-45ac-8437-f91f76ee3285 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-f2bb7806-8ae6-4e1a-b463-8d53649487d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:50.377358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:50.382857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:50.383804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:50.383804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:50.383804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:50.383804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 263/506] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:50.383804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cb54d-3298-496c-b754-b5514ad4348f None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.383804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cb54d-3298-496c-b754-b5514ad4348f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.383804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cb54d-3298-496c-b754-b5514ad4348f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-90268ff8-66f4-45ac-8437-f91f76ee3285 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:05:50.384857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:50.385699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:50.386578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:50.386578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:50.386578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:05:50.386578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:05:50.386578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:05:50.386578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:05:50.386578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:05:50.386578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:05:50.386578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:05:50.386578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:50.386578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 244/507] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:50.387113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90268ff8-66f4-45ac-8437-f91f76ee3285 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:50.404671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-fa4cb54d-3298-496c-b754-b5514ad4348f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:50.405889 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-fa4cb54d-3298-496c-b754-b5514ad4348f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:50.472013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 264/508] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:50.472797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cb54d-3298-496c-b754-b5514ad4348f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:50.492611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90268ff8-66f4-45ac-8437-f91f76ee3285 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:50.529329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613717e4-a560-4c06-8a65-91e9c8bb4c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:50.530384 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613717e4-a560-4c06-8a65-91e9c8bb4c1c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.530928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613717e4-a560-4c06-8a65-91e9c8bb4c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.531381 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613717e4-a560-4c06-8a65-91e9c8bb4c1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.544016 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613717e4-a560-4c06-8a65-91e9c8bb4c1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:50.546031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613717e4-a560-4c06-8a65-91e9c8bb4c1c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:50.556626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-613717e4-a560-4c06-8a65-91e9c8bb4c1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:50.563385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:50.564299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:50.565180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:50.565180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:50.565180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:50.565180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 245/509] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:50.565390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613717e4-a560-4c06-8a65-91e9c8bb4c1c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:50.577800 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cb54d-3298-496c-b754-b5514ad4348f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:50.598239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:50.618938 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5915d5-3980-4988-815b-ae1a6c3f9015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:50.619694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5915d5-3980-4988-815b-ae1a6c3f9015 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.620264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5915d5-3980-4988-815b-ae1a6c3f9015 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.620633 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5915d5-3980-4988-815b-ae1a6c3f9015 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.621720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5915d5-3980-4988-815b-ae1a6c3f9015 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:50.622347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5915d5-3980-4988-815b-ae1a6c3f9015 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:50.624276 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5915d5-3980-4988-815b-ae1a6c3f9015 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:50.640350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 265/510] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:50.641739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5915d5-3980-4988-815b-ae1a6c3f9015 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:50.684402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613717e4-a560-4c06-8a65-91e9c8bb4c1c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:50.722466 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96db709-5042-44a8-926b-24b7b656df77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:50.723883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96db709-5042-44a8-926b-24b7b656df77 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.724294 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96db709-5042-44a8-926b-24b7b656df77 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.724666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96db709-5042-44a8-926b-24b7b656df77 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.725401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96db709-5042-44a8-926b-24b7b656df77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:50.725800 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96db709-5042-44a8-926b-24b7b656df77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:50.727271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96db709-5042-44a8-926b-24b7b656df77 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:50.742076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 246/511] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:50.742904 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96db709-5042-44a8-926b-24b7b656df77 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:50.758785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5915d5-3980-4988-815b-ae1a6c3f9015 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:50.793575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba552da-822a-4c5e-b3b5-e195e9c987ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:50.794176 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba552da-822a-4c5e-b3b5-e195e9c987ec None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.794449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba552da-822a-4c5e-b3b5-e195e9c987ec None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.794717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba552da-822a-4c5e-b3b5-e195e9c987ec None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.795435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba552da-822a-4c5e-b3b5-e195e9c987ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:50.795732 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba552da-822a-4c5e-b3b5-e195e9c987ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:50.797292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba552da-822a-4c5e-b3b5-e195e9c987ec None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:50.814824 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 266/512] 10.222.0.85 () {56 vars in 1137 bytes} [Mon Aug 29 20:05:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:50.815752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba552da-822a-4c5e-b3b5-e195e9c987ec None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:50.818286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:50.837148 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:50.846309 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96db709-5042-44a8-926b-24b7b656df77 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:50.880071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35550711-dd2a-401c-9e84-e47a0063d01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:50.880695 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35550711-dd2a-401c-9e84-e47a0063d01c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.880971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35550711-dd2a-401c-9e84-e47a0063d01c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.881240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35550711-dd2a-401c-9e84-e47a0063d01c None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.881870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35550711-dd2a-401c-9e84-e47a0063d01c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:50.882306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35550711-dd2a-401c-9e84-e47a0063d01c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:50.883571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35550711-dd2a-401c-9e84-e47a0063d01c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:50.896507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 247/513] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:50.897416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35550711-dd2a-401c-9e84-e47a0063d01c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:50.917286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba552da-822a-4c5e-b3b5-e195e9c987ec None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:50.952107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef60cd-c716-45b6-8fdb-768b44ab9755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:50.952848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef60cd-c716-45b6-8fdb-768b44ab9755 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:50.953227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef60cd-c716-45b6-8fdb-768b44ab9755 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:50.953612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef60cd-c716-45b6-8fdb-768b44ab9755 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:50.960788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef60cd-c716-45b6-8fdb-768b44ab9755 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:50.962399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef60cd-c716-45b6-8fdb-768b44ab9755 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-b7ef60cd-c716-45b6-8fdb-768b44ab9755 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:05:50.975823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:50.976943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:50.977975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:50.977975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:50.977975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:05:50.977975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:05:50.977975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:05:50.977975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:05:50.977975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:05:50.977975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:05:50.977975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:05:50.977975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:50.977975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 267/514] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:50.977975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef60cd-c716-45b6-8fdb-768b44ab9755 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:51.003630 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35550711-dd2a-401c-9e84-e47a0063d01c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:51.036970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee6264-0635-42bb-81fe-718a5e8fc1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:51.037589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee6264-0635-42bb-81fe-718a5e8fc1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:51.037868 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee6264-0635-42bb-81fe-718a5e8fc1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:51.038150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee6264-0635-42bb-81fe-718a5e8fc1b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:51.040574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:51.049387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee6264-0635-42bb-81fe-718a5e8fc1b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:51.050840 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee6264-0635-42bb-81fe-718a5e8fc1b4 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-e1ee6264-0635-42bb-81fe-718a5e8fc1b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:51.064819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:51.065919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:51.066774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:51.066774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:51.066774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:51.066774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 248/515] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:51.066774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee6264-0635-42bb-81fe-718a5e8fc1b4 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:51.069472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:51.083044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef60cd-c716-45b6-8fdb-768b44ab9755 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:51.117507 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6f5d4-f074-4b77-b9d9-bc9f53752a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:51.118136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6f5d4-f074-4b77-b9d9-bc9f53752a1b None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:51.118419 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6f5d4-f074-4b77-b9d9-bc9f53752a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:51.118695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6f5d4-f074-4b77-b9d9-bc9f53752a1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:51.125267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6f5d4-f074-4b77-b9d9-bc9f53752a1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:51.126721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6f5d4-f074-4b77-b9d9-bc9f53752a1b None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-53e6f5d4-f074-4b77-b9d9-bc9f53752a1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:05:51.138418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:51.139616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:51.140489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:05:51.140489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:05:51.140489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:05:51.140489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:05:51.140489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:05:51.140489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:05:51.140489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:51.140489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:51.140489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 268/516] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:51.140489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6f5d4-f074-4b77-b9d9-bc9f53752a1b None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:51.168221 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee6264-0635-42bb-81fe-718a5e8fc1b4 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:51.205680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc174f-f9fd-46bc-92dc-e625de405ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:51.206281 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc174f-f9fd-46bc-92dc-e625de405ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:51.206557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc174f-f9fd-46bc-92dc-e625de405ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:51.206823 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc174f-f9fd-46bc-92dc-e625de405ad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:51.244026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6f5d4-f074-4b77-b9d9-bc9f53752a1b None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:51.265859 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:51.275778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc174f-f9fd-46bc-92dc-e625de405ad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:51.277408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc174f-f9fd-46bc-92dc-e625de405ad3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:51.277807 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98695676-ba5f-43a9-8047-510254a3eb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:51.278439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98695676-ba5f-43a9-8047-510254a3eb47 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:51.278715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98695676-ba5f-43a9-8047-510254a3eb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:51.278981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98695676-ba5f-43a9-8047-510254a3eb47 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:51.279280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:51.286930 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98695676-ba5f-43a9-8047-510254a3eb47 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:51.288432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98695676-ba5f-43a9-8047-510254a3eb47 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-98695676-ba5f-43a9-8047-510254a3eb47 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:05:51.300379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:51.301479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:51.302461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:51.302461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:51.302461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:05:51.302461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:05:51.302461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:05:51.302461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:05:51.302461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:05:51.302461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:05:51.302461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:05:51.302461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:51.303228 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 269/517] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:05:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:51.304032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98695676-ba5f-43a9-8047-510254a3eb47 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:51.379080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 249/518] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:51.379908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc174f-f9fd-46bc-92dc-e625de405ad3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:51.405734 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98695676-ba5f-43a9-8047-510254a3eb47 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:51.440612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b166445-c1cb-4e1a-9164-6f71f03e686f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:51.441224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b166445-c1cb-4e1a-9164-6f71f03e686f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:51.441504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b166445-c1cb-4e1a-9164-6f71f03e686f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:51.441776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b166445-c1cb-4e1a-9164-6f71f03e686f None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:51.442499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b166445-c1cb-4e1a-9164-6f71f03e686f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:51.442808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b166445-c1cb-4e1a-9164-6f71f03e686f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:51.444260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b166445-c1cb-4e1a-9164-6f71f03e686f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:51.462539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 270/519] 10.222.0.85 () {56 vars in 1137 bytes} [Mon Aug 29 20:05:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:51.463163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b166445-c1cb-4e1a-9164-6f71f03e686f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:51.484078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc174f-f9fd-46bc-92dc-e625de405ad3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:51.486131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:51.505833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:51.519051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae55b7-9360-41b7-bd9d-07381c1f8a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:51.526088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae55b7-9360-41b7-bd9d-07381c1f8a6d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:51.526088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae55b7-9360-41b7-bd9d-07381c1f8a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:51.526088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae55b7-9360-41b7-bd9d-07381c1f8a6d None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:51.526088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae55b7-9360-41b7-bd9d-07381c1f8a6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:51.526088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae55b7-9360-41b7-bd9d-07381c1f8a6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:51.526088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae55b7-9360-41b7-bd9d-07381c1f8a6d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:51.534633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 250/520] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:51.535476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae55b7-9360-41b7-bd9d-07381c1f8a6d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:51.567497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b166445-c1cb-4e1a-9164-6f71f03e686f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:51.601290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58c8ff-0264-45c8-a104-f2459a60bf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:51.601920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58c8ff-0264-45c8-a104-f2459a60bf8f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:51.602205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58c8ff-0264-45c8-a104-f2459a60bf8f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:51.602482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58c8ff-0264-45c8-a104-f2459a60bf8f None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:51.603174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58c8ff-0264-45c8-a104-f2459a60bf8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:51.603481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58c8ff-0264-45c8-a104-f2459a60bf8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:51.604989 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58c8ff-0264-45c8-a104-f2459a60bf8f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:51.626759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 271/521] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:05:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:51.627583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58c8ff-0264-45c8-a104-f2459a60bf8f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:51.637858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae55b7-9360-41b7-bd9d-07381c1f8a6d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:51.670373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8544d52-6b90-435b-bca0-831dac727e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:51.670971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8544d52-6b90-435b-bca0-831dac727e55 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:51.671251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8544d52-6b90-435b-bca0-831dac727e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:51.671518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8544d52-6b90-435b-bca0-831dac727e55 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:51.672791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8544d52-6b90-435b-bca0-831dac727e55 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:51.673095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8544d52-6b90-435b-bca0-831dac727e55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:51.674520 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8544d52-6b90-435b-bca0-831dac727e55 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:51.709895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 251/522] 10.222.0.85 () {56 vars in 1139 bytes} [Mon Aug 29 20:05:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:51.710720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8544d52-6b90-435b-bca0-831dac727e55 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:51.724936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:51.728739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58c8ff-0264-45c8-a104-f2459a60bf8f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:51.733296 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:51.762373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71eac86-e9ce-4f5b-b879-478abb0d535c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:51.762905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eac86-e9ce-4f5b-b879-478abb0d535c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:51.763191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eac86-e9ce-4f5b-b879-478abb0d535c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:51.763506 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eac86-e9ce-4f5b-b879-478abb0d535c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:51.774170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71eac86-e9ce-4f5b-b879-478abb0d535c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:51.775884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71eac86-e9ce-4f5b-b879-478abb0d535c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-b71eac86-e9ce-4f5b-b879-478abb0d535c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:51.790861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:51.792041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:51.792950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:51.792950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:51.792950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:51.792950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 272/523] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:51.792950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71eac86-e9ce-4f5b-b879-478abb0d535c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:51.815855 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8544d52-6b90-435b-bca0-831dac727e55 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:51.855744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd57f2-4ac6-490a-bb09-cb42d220f832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:51.856427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd57f2-4ac6-490a-bb09-cb42d220f832 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:51.856709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd57f2-4ac6-490a-bb09-cb42d220f832 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:51.856989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd57f2-4ac6-490a-bb09-cb42d220f832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:51.908407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71eac86-e9ce-4f5b-b879-478abb0d535c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:51.939654 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd57f2-4ac6-490a-bb09-cb42d220f832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:51.942057 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd57f2-4ac6-490a-bb09-cb42d220f832 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:51.949553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d06f4-6a1f-4504-8540-78672353fdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:51.950430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d06f4-6a1f-4504-8540-78672353fdb6 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:51.950430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d06f4-6a1f-4504-8540-78672353fdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:51.950747 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d06f4-6a1f-4504-8540-78672353fdb6 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:51.951458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d06f4-6a1f-4504-8540-78672353fdb6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:51.952940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d06f4-6a1f-4504-8540-78672353fdb6 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:51.965092 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:52.043784 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 252/524] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:52.044394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd57f2-4ac6-490a-bb09-cb42d220f832 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:52.077892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 273/525] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:05:51 2022] POST /identity/v3/users => generated 247 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:05:52.081153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d06f4-6a1f-4504-8540-78672353fdb6 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:52.097804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:52.165426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd57f2-4ac6-490a-bb09-cb42d220f832 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:52.193435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:52.207266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d06f4-6a1f-4504-8540-78672353fdb6 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:52.208804 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af70e7-155f-4de3-8f1f-3d7e655211cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:52.209491 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af70e7-155f-4de3-8f1f-3d7e655211cb None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:52.209758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af70e7-155f-4de3-8f1f-3d7e655211cb None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:52.210064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af70e7-155f-4de3-8f1f-3d7e655211cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:52.210796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af70e7-155f-4de3-8f1f-3d7e655211cb None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=805df3047fee45c7a702d72a5c082eb5) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:52.211103 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af70e7-155f-4de3-8f1f-3d7e655211cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:52.212594 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af70e7-155f-4de3-8f1f-3d7e655211cb None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:52.224224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 253/526] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:52 2022] GET /identity/v3/role_assignments?user.id=805df3047fee45c7a702d72a5c082eb5&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:52.224224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af70e7-155f-4de3-8f1f-3d7e655211cb None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:52.258427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a788878-9c50-4c9b-ab6d-444be760d80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:52.259153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a788878-9c50-4c9b-ab6d-444be760d80b None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:52.259458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a788878-9c50-4c9b-ab6d-444be760d80b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:52.259808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a788878-9c50-4c9b-ab6d-444be760d80b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:52.358597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a788878-9c50-4c9b-ab6d-444be760d80b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:52.359994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a788878-9c50-4c9b-ab6d-444be760d80b None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:52.365250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af70e7-155f-4de3-8f1f-3d7e655211cb None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:52.372623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:52.412513 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6df1cd-bea0-441e-ace9-45c1bd48125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:52.413552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6df1cd-bea0-441e-ace9-45c1bd48125e None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:52.413760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6df1cd-bea0-441e-ace9-45c1bd48125e None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:52.414047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6df1cd-bea0-441e-ace9-45c1bd48125e None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:52.414691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6df1cd-bea0-441e-ace9-45c1bd48125e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:52.414997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6df1cd-bea0-441e-ace9-45c1bd48125e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:52.416706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6df1cd-bea0-441e-ace9-45c1bd48125e None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:52.432532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 254/527] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:52.433746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6df1cd-bea0-441e-ace9-45c1bd48125e None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:52.488543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:52.503766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 274/528] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:52.503766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a788878-9c50-4c9b-ab6d-444be760d80b None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:52.554736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6df1cd-bea0-441e-ace9-45c1bd48125e None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:52.592334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a357e-f27f-4210-8a84-0fbfac15ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:52.593270 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a357e-f27f-4210-8a84-0fbfac15ba19 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:52.593732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a357e-f27f-4210-8a84-0fbfac15ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:52.594731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a357e-f27f-4210-8a84-0fbfac15ba19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:52.605919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a357e-f27f-4210-8a84-0fbfac15ba19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:52.607754 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a357e-f27f-4210-8a84-0fbfac15ba19 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-af8a357e-f27f-4210-8a84-0fbfac15ba19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:52.626773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:52.627742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:52.628627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:52.628627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:52.628627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:52.630454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 255/529] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:52.631517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a357e-f27f-4210-8a84-0fbfac15ba19 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:52.633345 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a788878-9c50-4c9b-ab6d-444be760d80b None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:52.645552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:52.680203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd3ea2-ca15-484e-ba71-b4bce402da61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:52.680406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd3ea2-ca15-484e-ba71-b4bce402da61 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:52.680642 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd3ea2-ca15-484e-ba71-b4bce402da61 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:52.680846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd3ea2-ca15-484e-ba71-b4bce402da61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:52.689043 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd3ea2-ca15-484e-ba71-b4bce402da61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:52.690547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd3ea2-ca15-484e-ba71-b4bce402da61 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-b4cd3ea2-ca15-484e-ba71-b4bce402da61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:05:52.702575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:52.703481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:52.704427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:05:52.704427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:05:52.704427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:05:52.704427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:05:52.704427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:05:52.704427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:05:52.704427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:52.704427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:52.708562 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 275/530] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:52.709427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd3ea2-ca15-484e-ba71-b4bce402da61 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:52.752445 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a357e-f27f-4210-8a84-0fbfac15ba19 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:52.788577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:52.790724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74facaf5-ad76-4b99-860a-fd35769b4ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:52.791505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74facaf5-ad76-4b99-860a-fd35769b4ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:52.791884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74facaf5-ad76-4b99-860a-fd35769b4ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:52.792318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74facaf5-ad76-4b99-860a-fd35769b4ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:52.804443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74facaf5-ad76-4b99-860a-fd35769b4ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:52.806141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74facaf5-ad76-4b99-860a-fd35769b4ff2 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-74facaf5-ad76-4b99-860a-fd35769b4ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:52.822427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:52.824738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:52.826552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:52.826552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:52.826552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:52.826552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 256/531] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:52.826552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74facaf5-ad76-4b99-860a-fd35769b4ff2 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:52.831528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd3ea2-ca15-484e-ba71-b4bce402da61 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:52.869262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b7cd2-0e73-4b11-992b-3d0146b9cf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:52.869785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b7cd2-0e73-4b11-992b-3d0146b9cf2f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:52.870058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b7cd2-0e73-4b11-992b-3d0146b9cf2f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:52.870328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b7cd2-0e73-4b11-992b-3d0146b9cf2f None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:52.870965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b7cd2-0e73-4b11-992b-3d0146b9cf2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:52.871262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b7cd2-0e73-4b11-992b-3d0146b9cf2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:52.873104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b7cd2-0e73-4b11-992b-3d0146b9cf2f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:52.887880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 276/532] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:52.888651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b7cd2-0e73-4b11-992b-3d0146b9cf2f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:52.923933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:52.947082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74facaf5-ad76-4b99-860a-fd35769b4ff2 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:52.989708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ed783-671e-4976-bd78-a7e160fccb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:52.990363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ed783-671e-4976-bd78-a7e160fccb56 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:52.990696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ed783-671e-4976-bd78-a7e160fccb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:52.990991 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ed783-671e-4976-bd78-a7e160fccb56 None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:52.992138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ed783-671e-4976-bd78-a7e160fccb56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:52.992468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ed783-671e-4976-bd78-a7e160fccb56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:52.994229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ed783-671e-4976-bd78-a7e160fccb56 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:53.011416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b7cd2-0e73-4b11-992b-3d0146b9cf2f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:53.018028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 257/533] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:05:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:53.018939 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ed783-671e-4976-bd78-a7e160fccb56 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:53.033539 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:53.051496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e930b4-a75e-4379-a687-aac98fd46f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:53.052472 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e930b4-a75e-4379-a687-aac98fd46f89 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:53.053035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e930b4-a75e-4379-a687-aac98fd46f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:53.053861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e930b4-a75e-4379-a687-aac98fd46f89 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:53.054805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e930b4-a75e-4379-a687-aac98fd46f89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:53.055338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e930b4-a75e-4379-a687-aac98fd46f89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:53.056936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e930b4-a75e-4379-a687-aac98fd46f89 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:53.070818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 277/534] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:53.072070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e930b4-a75e-4379-a687-aac98fd46f89 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:53.136934 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ed783-671e-4976-bd78-a7e160fccb56 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:53.174724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a1920-ffa2-4764-a7dc-11c91ba65eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:53.175790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a1920-ffa2-4764-a7dc-11c91ba65eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:53.176321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a1920-ffa2-4764-a7dc-11c91ba65eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:53.176799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a1920-ffa2-4764-a7dc-11c91ba65eb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:53.182639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:53.184432 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a1920-ffa2-4764-a7dc-11c91ba65eb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:53.186183 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a1920-ffa2-4764-a7dc-11c91ba65eb6 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:53.195469 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e930b4-a75e-4379-a687-aac98fd46f89 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-104a1920-ffa2-4764-a7dc-11c91ba65eb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:05:53.199371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:53.204857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:53.205699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:05:53.205699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:05:53.205699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:05:53.205699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:05:53.205699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:05:53.205699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:05:53.205699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:53.205699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:53.206103 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 258/535] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:53.207192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a1920-ffa2-4764-a7dc-11c91ba65eb6 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:53.236776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba3052-dbff-4306-b3d5-9046723af7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:53.237397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba3052-dbff-4306-b3d5-9046723af7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:53.237667 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba3052-dbff-4306-b3d5-9046723af7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:53.237940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba3052-dbff-4306-b3d5-9046723af7f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:53.238882 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba3052-dbff-4306-b3d5-9046723af7f1 None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=a7e768fdd1054e40b6b4910e39a6e533) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:53.239187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba3052-dbff-4306-b3d5-9046723af7f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:53.240661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba3052-dbff-4306-b3d5-9046723af7f1 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:53.246680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 278/536] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:53 2022] GET /identity/v3/role_assignments?user.id=a7e768fdd1054e40b6b4910e39a6e533&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:53.247557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba3052-dbff-4306-b3d5-9046723af7f1 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:53.329142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:53.400751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a1920-ffa2-4764-a7dc-11c91ba65eb6 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:53.451311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba3052-dbff-4306-b3d5-9046723af7f1 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:53.457439 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25e9ce-a2c9-45c6-b5de-ac27f8abe99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:53.458249 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25e9ce-a2c9-45c6-b5de-ac27f8abe99d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:53.458536 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25e9ce-a2c9-45c6-b5de-ac27f8abe99d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:53.458811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25e9ce-a2c9-45c6-b5de-ac27f8abe99d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:53.468339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25e9ce-a2c9-45c6-b5de-ac27f8abe99d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:53.470883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25e9ce-a2c9-45c6-b5de-ac27f8abe99d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-1e25e9ce-a2c9-45c6-b5de-ac27f8abe99d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:05:53.487177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:53.488067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:53.488941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:05:53.488941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:05:53.488941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:05:53.488941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:05:53.488941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:05:53.488941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:05:53.488941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:53.488941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:53.488941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 259/537] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:53.493966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25e9ce-a2c9-45c6-b5de-ac27f8abe99d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:53.500111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62c8a2-6aa4-4869-b9fd-7b9f4e38fc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:53.500846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62c8a2-6aa4-4869-b9fd-7b9f4e38fc60 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:53.501127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62c8a2-6aa4-4869-b9fd-7b9f4e38fc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:53.501400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62c8a2-6aa4-4869-b9fd-7b9f4e38fc60 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:53.502097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62c8a2-6aa4-4869-b9fd-7b9f4e38fc60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:53.502402 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62c8a2-6aa4-4869-b9fd-7b9f4e38fc60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:53.503831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62c8a2-6aa4-4869-b9fd-7b9f4e38fc60 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:53.542077 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 279/538] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:53.548081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62c8a2-6aa4-4869-b9fd-7b9f4e38fc60 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:53.594102 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:53.618839 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25e9ce-a2c9-45c6-b5de-ac27f8abe99d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:53.658121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb1f2f-0afb-46fb-872d-cc6022ec8f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:53.658973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb1f2f-0afb-46fb-872d-cc6022ec8f33 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:53.659399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb1f2f-0afb-46fb-872d-cc6022ec8f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:53.659835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb1f2f-0afb-46fb-872d-cc6022ec8f33 None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:53.660731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb1f2f-0afb-46fb-872d-cc6022ec8f33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:53.661184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb1f2f-0afb-46fb-872d-cc6022ec8f33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:53.663102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb1f2f-0afb-46fb-872d-cc6022ec8f33 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:53.677892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 260/539] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:53.678247 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:53.684624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb1f2f-0afb-46fb-872d-cc6022ec8f33 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:53.771101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62c8a2-6aa4-4869-b9fd-7b9f4e38fc60 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:53.816252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955c219-d8fe-4db7-b9bc-888ccbdd98d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:53.817334 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955c219-d8fe-4db7-b9bc-888ccbdd98d2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:53.817892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955c219-d8fe-4db7-b9bc-888ccbdd98d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:53.818383 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955c219-d8fe-4db7-b9bc-888ccbdd98d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:53.819263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955c219-d8fe-4db7-b9bc-888ccbdd98d2 None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=9d90a7ed44fd4a1fa94b9d44939df425) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:53.819939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955c219-d8fe-4db7-b9bc-888ccbdd98d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:53.821599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955c219-d8fe-4db7-b9bc-888ccbdd98d2 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:53.835258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 280/540] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:53 2022] GET /identity/v3/role_assignments?user.id=9d90a7ed44fd4a1fa94b9d44939df425&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:53.918954 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb1f2f-0afb-46fb-872d-cc6022ec8f33 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:53.980452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3fe991-d214-4e37-875f-241dc7253782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:53.981354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fe991-d214-4e37-875f-241dc7253782 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:53.981781 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fe991-d214-4e37-875f-241dc7253782 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:53.982233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fe991-d214-4e37-875f-241dc7253782 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:53.983130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3fe991-d214-4e37-875f-241dc7253782 None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=aa222b1c89dc4d2cb1686b4838c7908b) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:53.983596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3fe991-d214-4e37-875f-241dc7253782 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:53.985244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3fe991-d214-4e37-875f-241dc7253782 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:53.991833 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 261/541] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:53 2022] GET /identity/v3/role_assignments?user.id=aa222b1c89dc4d2cb1686b4838c7908b&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:54.274268 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd600f3e-52f8-455e-9932-5d002446ba3e None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:54.274609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd600f3e-52f8-455e-9932-5d002446ba3e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:54.274783 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd600f3e-52f8-455e-9932-5d002446ba3e None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:54.276523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 281/542] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:54.283338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86905ce-1896-44f9-a2b8-9ee8dfc10b86 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:54.283622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86905ce-1896-44f9-a2b8-9ee8dfc10b86 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:54.283898 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86905ce-1896-44f9-a2b8-9ee8dfc10b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:54.314370 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d86905ce-1896-44f9-a2b8-9ee8dfc10b86 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:54.315570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d86905ce-1896-44f9-a2b8-9ee8dfc10b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:54.345830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1ee91-bebe-4494-bce2-697fb41aecfa None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:54.346157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1ee91-bebe-4494-bce2-697fb41aecfa None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:54.346393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1ee91-bebe-4494-bce2-697fb41aecfa None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:54.348007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 282/543] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:54.353040 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee8129-ae05-404a-8371-65fb601eb1db None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:54.353327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee8129-ae05-404a-8371-65fb601eb1db None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:54.353605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee8129-ae05-404a-8371-65fb601eb1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:54.373266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-27ee8129-ae05-404a-8371-65fb601eb1db None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:54.374478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-27ee8129-ae05-404a-8371-65fb601eb1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:54.392546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:54.431433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 262/544] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:54.456797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 283/545] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:54.809231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86905ce-1896-44f9-a2b8-9ee8dfc10b86 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:54.815676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee8129-ae05-404a-8371-65fb601eb1db None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:54.825457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:54.978360 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86905ce-1896-44f9-a2b8-9ee8dfc10b86 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:55.024794 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee8129-ae05-404a-8371-65fb601eb1db None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:55.027443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9b902-a365-4ec6-9f2e-71b29be9ce4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:55.028072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9b902-a365-4ec6-9f2e-71b29be9ce4d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:55.028371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9b902-a365-4ec6-9f2e-71b29be9ce4d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:55.028697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9b902-a365-4ec6-9f2e-71b29be9ce4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:55.055397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9b902-a365-4ec6-9f2e-71b29be9ce4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-11a9b902-a365-4ec6-9f2e-71b29be9ce4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:55.061436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:55.062314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:55.063663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:55.063663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:55.063663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:55.063663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:55.063663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:55.063663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:55.063663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:55.063663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:55.063663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:55.063663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:55.063663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:55.063663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:55.068137 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 263/546] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:55.069624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9b902-a365-4ec6-9f2e-71b29be9ce4d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:55.116303 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96628a9c-e4a2-4322-bced-dedce0375069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:55.116946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96628a9c-e4a2-4322-bced-dedce0375069 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:55.117388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96628a9c-e4a2-4322-bced-dedce0375069 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:55.117665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96628a9c-e4a2-4322-bced-dedce0375069 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:55.140549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:55.143273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96628a9c-e4a2-4322-bced-dedce0375069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-96628a9c-e4a2-4322-bced-dedce0375069 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:55.148604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:55.155984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:55.156901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:55.156901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:55.156901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:55.156901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:55.156901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:55.156901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:55.156901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:55.156901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:55.156901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:55.156901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:55.156901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:55.156901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:55.156901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 284/547] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:55.165121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96628a9c-e4a2-4322-bced-dedce0375069 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:55.241763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:55.250472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9b902-a365-4ec6-9f2e-71b29be9ce4d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:55.285636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96628a9c-e4a2-4322-bced-dedce0375069 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:55.324186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e300f07-6aca-403b-a6c0-dc680065ff01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:55.324793 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e300f07-6aca-403b-a6c0-dc680065ff01 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:55.325061 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e300f07-6aca-403b-a6c0-dc680065ff01 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:55.325336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e300f07-6aca-403b-a6c0-dc680065ff01 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:55.325998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e300f07-6aca-403b-a6c0-dc680065ff01 None admin] RBAC: Adding query filter params (name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:55.326295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e300f07-6aca-403b-a6c0-dc680065ff01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:55.327697 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e300f07-6aca-403b-a6c0-dc680065ff01 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:55.338096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b64004-40a3-490c-9432-4a84d6f760bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:55.338714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b64004-40a3-490c-9432-4a84d6f760bb None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:55.339001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b64004-40a3-490c-9432-4a84d6f760bb None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:55.339278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b64004-40a3-490c-9432-4a84d6f760bb None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:55.339952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b64004-40a3-490c-9432-4a84d6f760bb None admin] RBAC: Adding query filter params (name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:55.340284 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b64004-40a3-490c-9432-4a84d6f760bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:55.341370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 285/548] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:05:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:55.341684 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b64004-40a3-490c-9432-4a84d6f760bb None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:55.342829 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e300f07-6aca-403b-a6c0-dc680065ff01 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:55.364621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 264/549] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:05:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:55.370248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b64004-40a3-490c-9432-4a84d6f760bb None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:55.475707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e300f07-6aca-403b-a6c0-dc680065ff01 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:55.508954 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:55.518367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76501b57-c2d2-4ffd-a47a-8e226100a562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:55.519365 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76501b57-c2d2-4ffd-a47a-8e226100a562 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:55.519804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76501b57-c2d2-4ffd-a47a-8e226100a562 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:55.527361 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76501b57-c2d2-4ffd-a47a-8e226100a562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:55.533164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76501b57-c2d2-4ffd-a47a-8e226100a562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:55.535074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76501b57-c2d2-4ffd-a47a-8e226100a562 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-76501b57-c2d2-4ffd-a47a-8e226100a562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:55.552714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:55.553688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:55.554523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:55.554523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:55.554523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:55.561336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 286/550] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:55.562858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76501b57-c2d2-4ffd-a47a-8e226100a562 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:55.591153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b64004-40a3-490c-9432-4a84d6f760bb None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:55.593829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:55.653540 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26cf6e-9f27-49e5-952a-80085eab04fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:55.654301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26cf6e-9f27-49e5-952a-80085eab04fa None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:55.654587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26cf6e-9f27-49e5-952a-80085eab04fa None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:55.654860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26cf6e-9f27-49e5-952a-80085eab04fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:55.682291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26cf6e-9f27-49e5-952a-80085eab04fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:55.683819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26cf6e-9f27-49e5-952a-80085eab04fa None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:55.691423 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76501b57-c2d2-4ffd-a47a-8e226100a562 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-9b26cf6e-9f27-49e5-952a-80085eab04fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:55.700403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:55.701631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:55.702671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:55.702671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:55.702671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:55.706240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 265/551] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:55.707761 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94b6a8-edb1-42eb-9a5d-3ecc2a3b4ca8 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:55.707761 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94b6a8-edb1-42eb-9a5d-3ecc2a3b4ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:55.708054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94b6a8-edb1-42eb-9a5d-3ecc2a3b4ca8 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:55.713431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 266/552] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:55.713431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94b6a8-edb1-42eb-9a5d-3ecc2a3b4ca8 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:55.734252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1afd79-e4a6-4234-bb9d-52a4c6acef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:55.735162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1afd79-e4a6-4234-bb9d-52a4c6acef6e None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:55.735597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1afd79-e4a6-4234-bb9d-52a4c6acef6e None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:55.736117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1afd79-e4a6-4234-bb9d-52a4c6acef6e None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:55.737140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1afd79-e4a6-4234-bb9d-52a4c6acef6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:55.737592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1afd79-e4a6-4234-bb9d-52a4c6acef6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:55.739207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1afd79-e4a6-4234-bb9d-52a4c6acef6e None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:55.753439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 287/553] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:55.755038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d8bdf-c33e-430c-9e9e-a537369bc0cb None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:55.755554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d8bdf-c33e-430c-9e9e-a537369bc0cb None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:55.756060 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d8bdf-c33e-430c-9e9e-a537369bc0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:55.776238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-2e0d8bdf-c33e-430c-9e9e-a537369bc0cb None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:55.777917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2e0d8bdf-c33e-430c-9e9e-a537369bc0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:55.817862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:55.818751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94b6a8-edb1-42eb-9a5d-3ecc2a3b4ca8 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:55.854704 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94958d75-96ee-4630-9136-5bc175aaced8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:55.855312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94958d75-96ee-4630-9136-5bc175aaced8 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:55.855598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94958d75-96ee-4630-9136-5bc175aaced8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:55.855868 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94958d75-96ee-4630-9136-5bc175aaced8 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:55.864388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94958d75-96ee-4630-9136-5bc175aaced8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:55.864388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94958d75-96ee-4630-9136-5bc175aaced8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:55.864388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94958d75-96ee-4630-9136-5bc175aaced8 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:55.864836 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 288/554] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:55.866077 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d8bdf-c33e-430c-9e9e-a537369bc0cb None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:55.871266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 267/555] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:55.876755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94958d75-96ee-4630-9136-5bc175aaced8 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:55.896719 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:55.985878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94958d75-96ee-4630-9136-5bc175aaced8 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:56.022269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52ff28-50b2-491f-9ad9-646ad71acacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:56.022823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d8bdf-c33e-430c-9e9e-a537369bc0cb None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:56.022939 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52ff28-50b2-491f-9ad9-646ad71acacd None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.023174 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52ff28-50b2-491f-9ad9-646ad71acacd None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.023476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52ff28-50b2-491f-9ad9-646ad71acacd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.040271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52ff28-50b2-491f-9ad9-646ad71acacd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:56.040271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52ff28-50b2-491f-9ad9-646ad71acacd None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-ed52ff28-50b2-491f-9ad9-646ad71acacd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:05:56.045601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:56.046512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:56.048039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:56.048039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:56.048039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:05:56.048039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:05:56.048039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:05:56.048039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:05:56.048039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:05:56.048039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:05:56.048039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:05:56.048039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:56.048039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 268/556] 10.222.0.85 () {56 vars in 1093 bytes} [Mon Aug 29 20:05:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:05:56.060204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52ff28-50b2-491f-9ad9-646ad71acacd None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:56.089849 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ced7f-3a7d-4952-a8a7-878f987346b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:56.091115 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ced7f-3a7d-4952-a8a7-878f987346b5 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.091548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ced7f-3a7d-4952-a8a7-878f987346b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.091984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ced7f-3a7d-4952-a8a7-878f987346b5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.103700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ced7f-3a7d-4952-a8a7-878f987346b5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:56.105384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ced7f-3a7d-4952-a8a7-878f987346b5 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-698ced7f-3a7d-4952-a8a7-878f987346b5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:05:56.132195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:56.133632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:56.134543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:56.134543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:56.134543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:05:56.134543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:05:56.134543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:05:56.134543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:05:56.134543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:05:56.134543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:05:56.134543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:05:56.134543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:56.134543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 289/557] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:05:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:56.140186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:56.140333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ced7f-3a7d-4952-a8a7-878f987346b5 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:56.160512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52ff28-50b2-491f-9ad9-646ad71acacd None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:56.197265 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb1c53-3ae5-4381-a69b-da6320168a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:56.197598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb1c53-3ae5-4381-a69b-da6320168a9e None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.197865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb1c53-3ae5-4381-a69b-da6320168a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.198148 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb1c53-3ae5-4381-a69b-da6320168a9e None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.198874 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb1c53-3ae5-4381-a69b-da6320168a9e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:56.199176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb1c53-3ae5-4381-a69b-da6320168a9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:56.200632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb1c53-3ae5-4381-a69b-da6320168a9e None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:56.210001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:56.219984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 269/558] 10.222.0.85 () {56 vars in 1133 bytes} [Mon Aug 29 20:05:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:05:56.220857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb1c53-3ae5-4381-a69b-da6320168a9e None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:56.307536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ced7f-3a7d-4952-a8a7-878f987346b5 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:56.335250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb1c53-3ae5-4381-a69b-da6320168a9e None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:56.349082 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ce726-db1f-491c-a531-93cb46fcd72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:56.349859 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ce726-db1f-491c-a531-93cb46fcd72e None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.350279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ce726-db1f-491c-a531-93cb46fcd72e None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.350619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ce726-db1f-491c-a531-93cb46fcd72e None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.351624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ce726-db1f-491c-a531-93cb46fcd72e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:56.352000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ce726-db1f-491c-a531-93cb46fcd72e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:56.353802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ce726-db1f-491c-a531-93cb46fcd72e None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:56.376699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a0a9d-d015-47b9-af30-6fc55dd5a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:56.377381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 290/559] 10.222.0.85 () {56 vars in 1135 bytes} [Mon Aug 29 20:05:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:56.377493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a0a9d-d015-47b9-af30-6fc55dd5a915 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.377707 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a0a9d-d015-47b9-af30-6fc55dd5a915 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.377973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a0a9d-d015-47b9-af30-6fc55dd5a915 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.378260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ce726-db1f-491c-a531-93cb46fcd72e None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:56.380472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:56.397347 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a0a9d-d015-47b9-af30-6fc55dd5a915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-e14a0a9d-d015-47b9-af30-6fc55dd5a915 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:56.403239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:56.404396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:56.405453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:56.405453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:56.405453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:56.405453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:56.405453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:56.405453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:56.405453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:56.405453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:56.405453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:56.405453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:56.405453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:56.405453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:56.408348 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 270/560] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:05:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:56.409392 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a0a9d-d015-47b9-af30-6fc55dd5a915 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:56.481696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:56.493602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ce726-db1f-491c-a531-93cb46fcd72e None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:56.512156 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a0a9d-d015-47b9-af30-6fc55dd5a915 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:56.527910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ad4ca-82f5-41ec-a062-623881c529fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:56.528592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ad4ca-82f5-41ec-a062-623881c529fe None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.528875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ad4ca-82f5-41ec-a062-623881c529fe None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.529275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ad4ca-82f5-41ec-a062-623881c529fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.539122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ad4ca-82f5-41ec-a062-623881c529fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:56.540593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ad4ca-82f5-41ec-a062-623881c529fe None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:56.546414 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c7aaa-f354-4f8d-addb-081673ef2918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:56.547060 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c7aaa-f354-4f8d-addb-081673ef2918 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.547342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c7aaa-f354-4f8d-addb-081673ef2918 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.548344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c7aaa-f354-4f8d-addb-081673ef2918 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-162ad4ca-82f5-41ec-a062-623881c529fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:56.555140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:56.556091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:56.557037 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:56.557037 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:56.557037 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:56.557037 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 291/561] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:56.557274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ad4ca-82f5-41ec-a062-623881c529fe None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:56.615762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:56.623275 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c7aaa-f354-4f8d-addb-081673ef2918 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:56.624912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c7aaa-f354-4f8d-addb-081673ef2918 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:56.662767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ad4ca-82f5-41ec-a062-623881c529fe None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:56.696628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870ace3-11ee-4212-ab04-9962f3fa1bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:56.697246 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870ace3-11ee-4212-ab04-9962f3fa1bfd None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.697720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870ace3-11ee-4212-ab04-9962f3fa1bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.698219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870ace3-11ee-4212-ab04-9962f3fa1bfd None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.699866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870ace3-11ee-4212-ab04-9962f3fa1bfd None admin] RBAC: Adding query filter params (name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:56.700207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870ace3-11ee-4212-ab04-9962f3fa1bfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:56.701607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870ace3-11ee-4212-ab04-9962f3fa1bfd None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:56.712332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 292/562] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:05:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:56.713666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921562d1-7bd6-48d1-b5d8-0099cfdbd0c8 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.713937 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921562d1-7bd6-48d1-b5d8-0099cfdbd0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.714204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921562d1-7bd6-48d1-b5d8-0099cfdbd0c8 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.715675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 293/563] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:56.716577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d056c6e-fde0-47c9-a4de-8f96c04baff2 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.717152 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d056c6e-fde0-47c9-a4de-8f96c04baff2 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.717299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d056c6e-fde0-47c9-a4de-8f96c04baff2 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.718690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 294/564] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:05:56.720991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d056c6e-fde0-47c9-a4de-8f96c04baff2 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:56.723960 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:56.730267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 271/565] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:05:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:05:56.731116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c7aaa-f354-4f8d-addb-081673ef2918 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:56.824804 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:56.825583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d056c6e-fde0-47c9-a4de-8f96c04baff2 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:56.831582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c7aaa-f354-4f8d-addb-081673ef2918 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:56.859978 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984e2b9-5e90-4a1a-b51b-4a78b2ef06a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:56.860666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984e2b9-5e90-4a1a-b51b-4a78b2ef06a5 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.860961 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984e2b9-5e90-4a1a-b51b-4a78b2ef06a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.861232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984e2b9-5e90-4a1a-b51b-4a78b2ef06a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.861868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984e2b9-5e90-4a1a-b51b-4a78b2ef06a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:56.862168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984e2b9-5e90-4a1a-b51b-4a78b2ef06a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:56.863534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984e2b9-5e90-4a1a-b51b-4a78b2ef06a5 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:56.865044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0844e-cf7d-4c04-aa3c-f8d90c2e13ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:56.865801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0844e-cf7d-4c04-aa3c-f8d90c2e13ab None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.866120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0844e-cf7d-4c04-aa3c-f8d90c2e13ab None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.866541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0844e-cf7d-4c04-aa3c-f8d90c2e13ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.875085 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 295/566] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:56.876295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc881b-8f09-4980-920d-db131f337b2a None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.876578 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc881b-8f09-4980-920d-db131f337b2a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.876851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc881b-8f09-4980-920d-db131f337b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.881510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0844e-cf7d-4c04-aa3c-f8d90c2e13ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:56.882963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0844e-cf7d-4c04-aa3c-f8d90c2e13ab None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:56.894270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-a8fc881b-8f09-4980-920d-db131f337b2a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:56.895454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a8fc881b-8f09-4980-920d-db131f337b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-3ba0844e-cf7d-4c04-aa3c-f8d90c2e13ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:56.899189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:56.900421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:56.901335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:56.901335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:56.901335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:56.901335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 272/567] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:56.901954 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7658ee1-0d6a-4ef4-9ac0-30419ffe4f67 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:56.902119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7658ee1-0d6a-4ef4-9ac0-30419ffe4f67 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:56.902375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7658ee1-0d6a-4ef4-9ac0-30419ffe4f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:56.920568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e7658ee1-0d6a-4ef4-9ac0-30419ffe4f67 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:05:56.921787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e7658ee1-0d6a-4ef4-9ac0-30419ffe4f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:05:56.955243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:56.966623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 296/568] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:56.967687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc881b-8f09-4980-920d-db131f337b2a None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:56.992651 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 273/569] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:05:56.993765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7658ee1-0d6a-4ef4-9ac0-30419ffe4f67 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:57.083751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc881b-8f09-4980-920d-db131f337b2a None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:57.104631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:57.117266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7658ee1-0d6a-4ef4-9ac0-30419ffe4f67 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:57.119347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091e1ec-5bd7-4ceb-85a9-67f8d6944141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:57.120606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091e1ec-5bd7-4ceb-85a9-67f8d6944141 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.121105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091e1ec-5bd7-4ceb-85a9-67f8d6944141 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.121603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091e1ec-5bd7-4ceb-85a9-67f8d6944141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.133310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091e1ec-5bd7-4ceb-85a9-67f8d6944141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:57.135065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091e1ec-5bd7-4ceb-85a9-67f8d6944141 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-9091e1ec-5bd7-4ceb-85a9-67f8d6944141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:57.151639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:57.152627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:57.153487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:57.153487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:57.153487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:57.155606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 297/570] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:57.156797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091e1ec-5bd7-4ceb-85a9-67f8d6944141 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:57.157465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710c7b5-efcf-4d1b-bff8-bd8eac2cef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:57.158097 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710c7b5-efcf-4d1b-bff8-bd8eac2cef2e None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.158377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710c7b5-efcf-4d1b-bff8-bd8eac2cef2e None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.158645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710c7b5-efcf-4d1b-bff8-bd8eac2cef2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.168554 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710c7b5-efcf-4d1b-bff8-bd8eac2cef2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:57.170496 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710c7b5-efcf-4d1b-bff8-bd8eac2cef2e None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-8710c7b5-efcf-4d1b-bff8-bd8eac2cef2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:05:57.185708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:57.186649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:57.187517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:05:57.187517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:05:57.187517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:05:57.187517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:05:57.187517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:05:57.187517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:05:57.187517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:57.187517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:57.192072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 274/571] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:57.193065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710c7b5-efcf-4d1b-bff8-bd8eac2cef2e None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:57.213432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:57.277032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091e1ec-5bd7-4ceb-85a9-67f8d6944141 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:57.312209 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710c7b5-efcf-4d1b-bff8-bd8eac2cef2e None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:57.314887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5a3df-80c8-4170-9bd1-410d12e333a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:57.315499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5a3df-80c8-4170-9bd1-410d12e333a9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.315772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5a3df-80c8-4170-9bd1-410d12e333a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.316048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5a3df-80c8-4170-9bd1-410d12e333a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.316703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5a3df-80c8-4170-9bd1-410d12e333a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:57.317000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5a3df-80c8-4170-9bd1-410d12e333a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:57.318357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5a3df-80c8-4170-9bd1-410d12e333a9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:57.330372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 298/572] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:57.331173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5a3df-80c8-4170-9bd1-410d12e333a9 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:57.354441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db69cc6-1064-43fe-b1bf-80b841ae0e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:57.355056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db69cc6-1064-43fe-b1bf-80b841ae0e36 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.355338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db69cc6-1064-43fe-b1bf-80b841ae0e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.355613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db69cc6-1064-43fe-b1bf-80b841ae0e36 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.356293 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db69cc6-1064-43fe-b1bf-80b841ae0e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:57.356598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db69cc6-1064-43fe-b1bf-80b841ae0e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:57.357962 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db69cc6-1064-43fe-b1bf-80b841ae0e36 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:57.361001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:57.370688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 275/573] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:57.371432 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db69cc6-1064-43fe-b1bf-80b841ae0e36 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:57.435032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5a3df-80c8-4170-9bd1-410d12e333a9 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:57.437623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:57.471276 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8aacef-e7f6-4238-80b7-2f601e07ce86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:57.472241 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8aacef-e7f6-4238-80b7-2f601e07ce86 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.472880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8aacef-e7f6-4238-80b7-2f601e07ce86 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.473152 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8aacef-e7f6-4238-80b7-2f601e07ce86 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.473562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db69cc6-1064-43fe-b1bf-80b841ae0e36 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:57.473869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8aacef-e7f6-4238-80b7-2f601e07ce86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:57.474181 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8aacef-e7f6-4238-80b7-2f601e07ce86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:57.475603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8aacef-e7f6-4238-80b7-2f601e07ce86 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:57.487005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 299/574] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:57.487796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8aacef-e7f6-4238-80b7-2f601e07ce86 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:57.507834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9a1c2-9c37-48a3-82ea-bc01514c89eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:57.508494 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9a1c2-9c37-48a3-82ea-bc01514c89eb None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.508779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9a1c2-9c37-48a3-82ea-bc01514c89eb None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.509187 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9a1c2-9c37-48a3-82ea-bc01514c89eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.519742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9a1c2-9c37-48a3-82ea-bc01514c89eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-3be9a1c2-9c37-48a3-82ea-bc01514c89eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:57.525566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:57.526589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 276/575] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:05:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:57.527434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9a1c2-9c37-48a3-82ea-bc01514c89eb None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:57.586364 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:57.594671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8aacef-e7f6-4238-80b7-2f601e07ce86 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:57.632414 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9a1c2-9c37-48a3-82ea-bc01514c89eb None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:57.633500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b3954-a606-4c29-bdd7-cf5347b01da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:57.634136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b3954-a606-4c29-bdd7-cf5347b01da7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.634417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b3954-a606-4c29-bdd7-cf5347b01da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.634692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b3954-a606-4c29-bdd7-cf5347b01da7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.645533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b3954-a606-4c29-bdd7-cf5347b01da7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-a15b3954-a606-4c29-bdd7-cf5347b01da7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:05:57.651158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:57.652124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:57.652994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:57.652994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:57.652994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:57.652994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:57.652994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:05:57.652994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:05:57.652994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:05:57.652994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:05:57.652994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:05:57.652994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:05:57.652994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:05:57.652994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:57.652994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 300/576] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:05:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:57.653678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b3954-a606-4c29-bdd7-cf5347b01da7 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:57.667637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daacf1d-f8f1-44bc-97c8-b6878394d2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:57.668297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daacf1d-f8f1-44bc-97c8-b6878394d2bd None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.668578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daacf1d-f8f1-44bc-97c8-b6878394d2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.668850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daacf1d-f8f1-44bc-97c8-b6878394d2bd None admin] PATH_INFO: `/v3/users/glance` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.677040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daacf1d-f8f1-44bc-97c8-b6878394d2bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:57.678482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daacf1d-f8f1-44bc-97c8-b6878394d2bd None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:57.678684 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-4daacf1d-f8f1-44bc-97c8-b6878394d2bd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:05:57.695317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:57.702531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:57.704029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:57.704029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:57.704029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:05:57.704029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:05:57.704029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:05:57.704029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:05:57.704029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:05:57.704029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:05:57.704029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:05:57.704029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:57.704029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 277/577] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:57.704029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daacf1d-f8f1-44bc-97c8-b6878394d2bd None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:57.756437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b3954-a606-4c29-bdd7-cf5347b01da7 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:57.792723 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232243df-4626-47a2-ae0b-8eb7dfb30571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:57.793358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232243df-4626-47a2-ae0b-8eb7dfb30571 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.793642 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232243df-4626-47a2-ae0b-8eb7dfb30571 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.793913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232243df-4626-47a2-ae0b-8eb7dfb30571 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.800513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232243df-4626-47a2-ae0b-8eb7dfb30571 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:57.801521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daacf1d-f8f1-44bc-97c8-b6878394d2bd None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:57.801960 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232243df-4626-47a2-ae0b-8eb7dfb30571 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-232243df-4626-47a2-ae0b-8eb7dfb30571 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:05:57.814188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:57.815139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:57.816101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:05:57.816101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:05:57.816101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:05:57.816101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:05:57.816101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:05:57.816101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:05:57.816101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:57.816101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:57.816101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 301/578] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:57.816101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232243df-4626-47a2-ae0b-8eb7dfb30571 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:57.820829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:57.841250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd29252-c67e-4574-9d76-6be64cadc0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:57.842052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd29252-c67e-4574-9d76-6be64cadc0c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.842328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd29252-c67e-4574-9d76-6be64cadc0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.842601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd29252-c67e-4574-9d76-6be64cadc0c2 None admin] PATH_INFO: `/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/805df3047fee45c7a702d72a5c082eb5/roles/cd65aca5e0074973a10adcd1c9a180dc` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.861180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd29252-c67e-4574-9d76-6be64cadc0c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=11372fe8994543b4a19773c0e0bee53e, user_id=805df3047fee45c7a702d72a5c082eb5, role_id=cd65aca5e0074973a10adcd1c9a180dc)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:57.862623 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd29252-c67e-4574-9d76-6be64cadc0c2 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:57.905784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:57.921066 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232243df-4626-47a2-ae0b-8eb7dfb30571 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:57.939846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 278/579] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:05:57 2022] PUT /identity/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/805df3047fee45c7a702d72a5c082eb5/roles/cd65aca5e0074973a10adcd1c9a180dc => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:05:57.940943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd29252-c67e-4574-9d76-6be64cadc0c2 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:57.956154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e154041-cebb-4ea5-a998-6f06aebf203d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:57.956834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e154041-cebb-4ea5-a998-6f06aebf203d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:57.957090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e154041-cebb-4ea5-a998-6f06aebf203d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:57.957366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e154041-cebb-4ea5-a998-6f06aebf203d None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:57.958054 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e154041-cebb-4ea5-a998-6f06aebf203d None admin] RBAC: Adding query filter params (name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:57.958357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e154041-cebb-4ea5-a998-6f06aebf203d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:57.959787 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e154041-cebb-4ea5-a998-6f06aebf203d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:58.048107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 302/580] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:05:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:58.048107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e154041-cebb-4ea5-a998-6f06aebf203d None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:58.060226 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd29252-c67e-4574-9d76-6be64cadc0c2 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:58.098018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7851f-1db7-4e62-b58d-8b930530ded3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:58.098689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7851f-1db7-4e62-b58d-8b930530ded3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.098966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7851f-1db7-4e62-b58d-8b930530ded3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.099237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7851f-1db7-4e62-b58d-8b930530ded3 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.099882 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7851f-1db7-4e62-b58d-8b930530ded3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:58.100195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7851f-1db7-4e62-b58d-8b930530ded3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:58.101569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7851f-1db7-4e62-b58d-8b930530ded3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:58.111165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 279/581] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:05:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:58.111729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7851f-1db7-4e62-b58d-8b930530ded3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:58.144509 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:58.158988 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e154041-cebb-4ea5-a998-6f06aebf203d None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:58.195270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d9d9d-b580-41e3-8b24-2a405f9ecc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:58.195897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d9d9d-b580-41e3-8b24-2a405f9ecc5b None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.198008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d9d9d-b580-41e3-8b24-2a405f9ecc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.198008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d9d9d-b580-41e3-8b24-2a405f9ecc5b None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.198008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d9d9d-b580-41e3-8b24-2a405f9ecc5b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:58.198008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d9d9d-b580-41e3-8b24-2a405f9ecc5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:58.199177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d9d9d-b580-41e3-8b24-2a405f9ecc5b None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:58.221634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 303/582] 10.222.0.85 () {56 vars in 1137 bytes} [Mon Aug 29 20:05:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:58.221634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d9d9d-b580-41e3-8b24-2a405f9ecc5b None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:58.225062 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:58.229294 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7851f-1db7-4e62-b58d-8b930530ded3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:58.268310 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1df4a7-c748-48bd-8863-51d56473ee0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:58.268927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1df4a7-c748-48bd-8863-51d56473ee0b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.269200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1df4a7-c748-48bd-8863-51d56473ee0b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.269466 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1df4a7-c748-48bd-8863-51d56473ee0b None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.270152 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1df4a7-c748-48bd-8863-51d56473ee0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:58.270565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1df4a7-c748-48bd-8863-51d56473ee0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:58.272137 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1df4a7-c748-48bd-8863-51d56473ee0b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:58.285873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 280/583] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:58.286757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1df4a7-c748-48bd-8863-51d56473ee0b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:58.332554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d9d9d-b580-41e3-8b24-2a405f9ecc5b None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:58.369824 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea1dc5-8656-4a68-b393-75ffcdd266cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:58.371209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea1dc5-8656-4a68-b393-75ffcdd266cd None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.371700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea1dc5-8656-4a68-b393-75ffcdd266cd None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.372537 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea1dc5-8656-4a68-b393-75ffcdd266cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.383574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea1dc5-8656-4a68-b393-75ffcdd266cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:58.383762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:58.385577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea1dc5-8656-4a68-b393-75ffcdd266cd None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:58.397492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1df4a7-c748-48bd-8863-51d56473ee0b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-4bea1dc5-8656-4a68-b393-75ffcdd266cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:58.401769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:58.402692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:58.403565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:58.403565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:58.403565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:58.403565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 304/584] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:58.403800 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea1dc5-8656-4a68-b393-75ffcdd266cd None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:58.438477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2829981-b967-4692-9d18-7fa19e61b6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:58.439034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2829981-b967-4692-9d18-7fa19e61b6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.439297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2829981-b967-4692-9d18-7fa19e61b6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.439571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2829981-b967-4692-9d18-7fa19e61b6e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.450135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2829981-b967-4692-9d18-7fa19e61b6e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:58.451583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2829981-b967-4692-9d18-7fa19e61b6e8 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-d2829981-b967-4692-9d18-7fa19e61b6e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:58.467544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:58.468601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:58.469880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:58.469880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:58.469880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:58.469880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 281/585] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:58.469880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2829981-b967-4692-9d18-7fa19e61b6e8 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:58.471306 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:58.517955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea1dc5-8656-4a68-b393-75ffcdd266cd None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:58.552221 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da49c8-835d-420c-9883-a37692f7e8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:58.552866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da49c8-835d-420c-9883-a37692f7e8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.553145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da49c8-835d-420c-9883-a37692f7e8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.553423 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da49c8-835d-420c-9883-a37692f7e8d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.565939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da49c8-835d-420c-9883-a37692f7e8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:58.567388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da49c8-835d-420c-9883-a37692f7e8d8 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:58.577056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2829981-b967-4692-9d18-7fa19e61b6e8 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-e4da49c8-835d-420c-9883-a37692f7e8d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:58.582048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:58.582990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:58.583861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:58.583861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:58.583861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:58.583861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 305/586] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:58.584463 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da49c8-835d-420c-9883-a37692f7e8d8 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:58.625470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3c121-99f6-4f24-b299-5e49e55bb6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:58.626101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3c121-99f6-4f24-b299-5e49e55bb6a9 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.626364 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3c121-99f6-4f24-b299-5e49e55bb6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.626640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3c121-99f6-4f24-b299-5e49e55bb6a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.627378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3c121-99f6-4f24-b299-5e49e55bb6a9 None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=d52f84245a2142e69c309aa709a7d921) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:58.627693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3c121-99f6-4f24-b299-5e49e55bb6a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:58.629230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3c121-99f6-4f24-b299-5e49e55bb6a9 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:58.634905 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 282/587] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:05:58 2022] GET /identity/v3/role_assignments?user.id=d52f84245a2142e69c309aa709a7d921&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:58.635683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3c121-99f6-4f24-b299-5e49e55bb6a9 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:58.648054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:58.733036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da49c8-835d-420c-9883-a37692f7e8d8 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:58.747393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:58.778565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3c121-99f6-4f24-b299-5e49e55bb6a9 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:58.790584 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809686f7-ab23-4e94-96b7-1d925f17e283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:58.791935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809686f7-ab23-4e94-96b7-1d925f17e283 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.792420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809686f7-ab23-4e94-96b7-1d925f17e283 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.793226 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809686f7-ab23-4e94-96b7-1d925f17e283 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.794371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809686f7-ab23-4e94-96b7-1d925f17e283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:58.794929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809686f7-ab23-4e94-96b7-1d925f17e283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:58.797457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809686f7-ab23-4e94-96b7-1d925f17e283 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:58.817328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 306/588] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:58.817328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809686f7-ab23-4e94-96b7-1d925f17e283 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:58.827968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f102fa2-2961-422c-9ccb-ec33a2c7ccb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:58.829107 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f102fa2-2961-422c-9ccb-ec33a2c7ccb4 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.829107 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f102fa2-2961-422c-9ccb-ec33a2c7ccb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.829495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f102fa2-2961-422c-9ccb-ec33a2c7ccb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.830240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f102fa2-2961-422c-9ccb-ec33a2c7ccb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:58.830621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f102fa2-2961-422c-9ccb-ec33a2c7ccb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:58.832056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f102fa2-2961-422c-9ccb-ec33a2c7ccb4 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:58.850169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 283/589] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:58.850169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f102fa2-2961-422c-9ccb-ec33a2c7ccb4 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:58.920495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:58.949876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809686f7-ab23-4e94-96b7-1d925f17e283 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:58.973377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f102fa2-2961-422c-9ccb-ec33a2c7ccb4 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:58.990304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff34cf-b7c6-4d8f-a59e-a69d2a140d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:58.991219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff34cf-b7c6-4d8f-a59e-a69d2a140d42 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:58.991677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff34cf-b7c6-4d8f-a59e-a69d2a140d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:58.992253 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff34cf-b7c6-4d8f-a59e-a69d2a140d42 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:58.993237 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff34cf-b7c6-4d8f-a59e-a69d2a140d42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:58.993747 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff34cf-b7c6-4d8f-a59e-a69d2a140d42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:58.995303 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff34cf-b7c6-4d8f-a59e-a69d2a140d42 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:59.012600 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 307/590] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:59.013849 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff34cf-b7c6-4d8f-a59e-a69d2a140d42 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:59.017390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb84c4d-e931-4c12-a2a3-ef4ade2b2cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:59.018455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84c4d-e931-4c12-a2a3-ef4ade2b2cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:59.018810 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84c4d-e931-4c12-a2a3-ef4ade2b2cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:59.019158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84c4d-e931-4c12-a2a3-ef4ade2b2cd2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:59.029817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb84c4d-e931-4c12-a2a3-ef4ade2b2cd2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:59.032903 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb84c4d-e931-4c12-a2a3-ef4ade2b2cd2 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:59.050120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-1cb84c4d-e931-4c12-a2a3-ef4ade2b2cd2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:05:59.052693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:59.053613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:59.054589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:59.054589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:59.054589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:05:59.054589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:05:59.054589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:05:59.054589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:05:59.054589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:05:59.054589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:05:59.054589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:05:59.054589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:59.056248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 284/591] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:05:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:59.057274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb84c4d-e931-4c12-a2a3-ef4ade2b2cd2 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:59.146683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff34cf-b7c6-4d8f-a59e-a69d2a140d42 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:59.181249 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb84c4d-e931-4c12-a2a3-ef4ade2b2cd2 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:59.183914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb62ba0-497f-403c-b8d2-b2ec95d734a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:59.185114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb62ba0-497f-403c-b8d2-b2ec95d734a0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:59.185653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb62ba0-497f-403c-b8d2-b2ec95d734a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:59.186147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb62ba0-497f-403c-b8d2-b2ec95d734a0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:59.194510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb62ba0-497f-403c-b8d2-b2ec95d734a0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:59.196326 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb62ba0-497f-403c-b8d2-b2ec95d734a0 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:59.207991 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-8fb62ba0-497f-403c-b8d2-b2ec95d734a0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:05:59.210442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:59.211372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:59.212347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:59.212347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:59.212347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:05:59.212347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:05:59.212347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:05:59.212347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:05:59.212347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:05:59.212347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:05:59.212347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:05:59.212347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:59.219705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0790e25-446e-494c-9d07-43e7c7fc849d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:59.220003 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 308/592] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:05:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:59.220266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0790e25-446e-494c-9d07-43e7c7fc849d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:59.220551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0790e25-446e-494c-9d07-43e7c7fc849d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:59.220827 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0790e25-446e-494c-9d07-43e7c7fc849d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:59.221877 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb62ba0-497f-403c-b8d2-b2ec95d734a0 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:59.229043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0790e25-446e-494c-9d07-43e7c7fc849d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:59.230503 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0790e25-446e-494c-9d07-43e7c7fc849d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-a0790e25-446e-494c-9d07-43e7c7fc849d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:05:59.245111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:59.246049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:59.246923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:05:59.246923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:05:59.246923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:05:59.246923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:05:59.246923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:05:59.246923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:05:59.246923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:05:59.246923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:59.246923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 285/593] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:05:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:59.246923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0790e25-446e-494c-9d07-43e7c7fc849d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:59.345640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:59.358987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb62ba0-497f-403c-b8d2-b2ec95d734a0 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:59.376139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0790e25-446e-494c-9d07-43e7c7fc849d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:59.394887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55430c45-6daf-4c15-b5e4-e404d5d5ede0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:59.395882 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55430c45-6daf-4c15-b5e4-e404d5d5ede0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:59.396421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55430c45-6daf-4c15-b5e4-e404d5d5ede0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:59.396900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55430c45-6daf-4c15-b5e4-e404d5d5ede0 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:59.397829 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55430c45-6daf-4c15-b5e4-e404d5d5ede0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:59.398340 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55430c45-6daf-4c15-b5e4-e404d5d5ede0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:59.400034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55430c45-6daf-4c15-b5e4-e404d5d5ede0 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:59.419979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ea4cc-9ca9-4f19-9906-e16906b238e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:59.420986 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ea4cc-9ca9-4f19-9906-e16906b238e6 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:59.421469 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ea4cc-9ca9-4f19-9906-e16906b238e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:59.421974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ea4cc-9ca9-4f19-9906-e16906b238e6 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:59.422910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ea4cc-9ca9-4f19-9906-e16906b238e6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:59.423445 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ea4cc-9ca9-4f19-9906-e16906b238e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:59.424895 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 309/594] 10.222.0.85 () {56 vars in 1137 bytes} [Mon Aug 29 20:05:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:59.425701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ea4cc-9ca9-4f19-9906-e16906b238e6 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:59.425811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55430c45-6daf-4c15-b5e4-e404d5d5ede0 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:59.450267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 286/595] 10.222.0.85 () {56 vars in 1139 bytes} [Mon Aug 29 20:05:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:59.451560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ea4cc-9ca9-4f19-9906-e16906b238e6 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:59.499446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:59.564885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55430c45-6daf-4c15-b5e4-e404d5d5ede0 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:59.565672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ea4cc-9ca9-4f19-9906-e16906b238e6 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:59.603576 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211efbb-3910-41d0-95e4-af4290c9a218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:59.604594 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211efbb-3910-41d0-95e4-af4290c9a218 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:59.607174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211efbb-3910-41d0-95e4-af4290c9a218 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:59.607690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211efbb-3910-41d0-95e4-af4290c9a218 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:59.609099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211efbb-3910-41d0-95e4-af4290c9a218 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:05:59.610184 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211efbb-3910-41d0-95e4-af4290c9a218 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:59.614477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211efbb-3910-41d0-95e4-af4290c9a218 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:59.616861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaafbdb-8275-4c3f-81e8-37275c5c8bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:59.617508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaafbdb-8275-4c3f-81e8-37275c5c8bfd None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:59.617790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaafbdb-8275-4c3f-81e8-37275c5c8bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:59.618065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaafbdb-8275-4c3f-81e8-37275c5c8bfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:59.633437 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaafbdb-8275-4c3f-81e8-37275c5c8bfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:59.633437 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaafbdb-8275-4c3f-81e8-37275c5c8bfd None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:59.634158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 310/596] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:05:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:05:59.635273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211efbb-3910-41d0-95e4-af4290c9a218 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:59.647057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-cfaafbdb-8275-4c3f-81e8-37275c5c8bfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:59.654468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:59.655444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:59.661925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:59.661925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:59.661925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:05:59.661925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 287/597] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:59.661925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaafbdb-8275-4c3f-81e8-37275c5c8bfd None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:59.746907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211efbb-3910-41d0-95e4-af4290c9a218 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:59.776375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:59.784859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaafbdb-8275-4c3f-81e8-37275c5c8bfd None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:59.794400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61783fc6-241e-4732-b2fb-70f9e762d05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:59.795131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61783fc6-241e-4732-b2fb-70f9e762d05a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:59.795485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61783fc6-241e-4732-b2fb-70f9e762d05a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:59.795758 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61783fc6-241e-4732-b2fb-70f9e762d05a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:59.811676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61783fc6-241e-4732-b2fb-70f9e762d05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:59.813761 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61783fc6-241e-4732-b2fb-70f9e762d05a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:59.827410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d592b1-c725-486a-b523-871a8629b825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:59.828983 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d592b1-c725-486a-b523-871a8629b825 None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:59.829467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d592b1-c725-486a-b523-871a8629b825 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:05:59.829942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d592b1-c725-486a-b523-871a8629b825 None admin] PATH_INFO: `/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/a7e768fdd1054e40b6b4910e39a6e533/roles/cd65aca5e0074973a10adcd1c9a180dc` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-61783fc6-241e-4732-b2fb-70f9e762d05a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:05:59.834529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:05:59.835474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:05:59.836398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:05:59.836398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:05:59.836398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:05:59.836398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 311/598] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:05:59.841594 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61783fc6-241e-4732-b2fb-70f9e762d05a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:59.856344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d592b1-c725-486a-b523-871a8629b825 None admin] RBAC: Authorizing `identity:create_grant(project_id=11372fe8994543b4a19773c0e0bee53e, user_id=a7e768fdd1054e40b6b4910e39a6e533, role_id=cd65aca5e0074973a10adcd1c9a180dc)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:05:59.858587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d592b1-c725-486a-b523-871a8629b825 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:05:59.907902 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:05:59.948718 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61783fc6-241e-4732-b2fb-70f9e762d05a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:05:59.963512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 288/599] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:05:59 2022] PUT /identity/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/a7e768fdd1054e40b6b4910e39a6e533/roles/cd65aca5e0074973a10adcd1c9a180dc => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:05:59.964474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d592b1-c725-486a-b523-871a8629b825 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:05:59.998320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b294bf-d585-4c42-a77a-81892142a33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:05:59.999253 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b294bf-d585-4c42-a77a-81892142a33a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:05:59.999727 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b294bf-d585-4c42-a77a-81892142a33a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:00.000290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b294bf-d585-4c42-a77a-81892142a33a None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:00.000957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b294bf-d585-4c42-a77a-81892142a33a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:00.001263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b294bf-d585-4c42-a77a-81892142a33a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:00.002892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b294bf-d585-4c42-a77a-81892142a33a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:00.017494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 312/600] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:00.019004 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0238cb-25ed-420c-b215-eb8da630ffd4 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:00.019268 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0238cb-25ed-420c-b215-eb8da630ffd4 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:00.019553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0238cb-25ed-420c-b215-eb8da630ffd4 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:00.020976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 313/601] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:00.026041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0238cb-25ed-420c-b215-eb8da630ffd4 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:00.136930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d592b1-c725-486a-b523-871a8629b825 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:00.151933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0238cb-25ed-420c-b215-eb8da630ffd4 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:00.174182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:00.182936 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a257d-f2df-4274-a596-22b3b74d188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:00.189931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a257d-f2df-4274-a596-22b3b74d188d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:00.189931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a257d-f2df-4274-a596-22b3b74d188d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:00.189931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a257d-f2df-4274-a596-22b3b74d188d None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:00.189931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a257d-f2df-4274-a596-22b3b74d188d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:00.189931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a257d-f2df-4274-a596-22b3b74d188d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:00.189931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a257d-f2df-4274-a596-22b3b74d188d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:00.195289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd0902-c138-4d91-9fdc-dde7650f3e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:00.195933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd0902-c138-4d91-9fdc-dde7650f3e88 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:00.196435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd0902-c138-4d91-9fdc-dde7650f3e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:00.196747 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd0902-c138-4d91-9fdc-dde7650f3e88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:00.203958 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd0902-c138-4d91-9fdc-dde7650f3e88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:00.205452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd0902-c138-4d91-9fdc-dde7650f3e88 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:00.209883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 289/602] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:00.215524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b1011-1c4c-4af1-af02-39880a4fa8db None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:00.216140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b1011-1c4c-4af1-af02-39880a4fa8db None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:00.216551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b1011-1c4c-4af1-af02-39880a4fa8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-cddd0902-c138-4d91-9fdc-dde7650f3e88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:00.225499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:00.226503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:00.227530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:00.227530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:00.227530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:00.227530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:00.227530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:00.227530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:00.227530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:00.227530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:00.227530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 314/603] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:00.228097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd0902-c138-4d91-9fdc-dde7650f3e88 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:00.230896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:00.243178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-4e3b1011-1c4c-4af1-af02-39880a4fa8db None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:00.246107 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-4e3b1011-1c4c-4af1-af02-39880a4fa8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:00.325815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 290/604] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:06:00.332458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b1011-1c4c-4af1-af02-39880a4fa8db None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:00.357392 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd0902-c138-4d91-9fdc-dde7650f3e88 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:00.408384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67775ee1-d23d-42b5-a129-86a5884c98b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:00.410534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67775ee1-d23d-42b5-a129-86a5884c98b2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:00.410867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67775ee1-d23d-42b5-a129-86a5884c98b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:00.411141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67775ee1-d23d-42b5-a129-86a5884c98b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:00.426712 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67775ee1-d23d-42b5-a129-86a5884c98b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:00.428186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67775ee1-d23d-42b5-a129-86a5884c98b2 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-67775ee1-d23d-42b5-a129-86a5884c98b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:00.458139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:00.459267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:00.460290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:00.460290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:00.460290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:00.460290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:00.460290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:00.460290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:00.460290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:00.460290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:00.460731 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 315/605] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:00.466434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67775ee1-d23d-42b5-a129-86a5884c98b2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:00.511281 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b1011-1c4c-4af1-af02-39880a4fa8db None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:00.528511 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:00.582174 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019806df-f197-459e-8398-4c08384d85bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:00.582785 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019806df-f197-459e-8398-4c08384d85bd None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:00.583060 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019806df-f197-459e-8398-4c08384d85bd None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:00.583329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019806df-f197-459e-8398-4c08384d85bd None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:00.584022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019806df-f197-459e-8398-4c08384d85bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:00.584356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019806df-f197-459e-8398-4c08384d85bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:00.590439 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019806df-f197-459e-8398-4c08384d85bd None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:00.595497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:00.612767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 291/606] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:00.617671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019806df-f197-459e-8398-4c08384d85bd None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:00.651853 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67775ee1-d23d-42b5-a129-86a5884c98b2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:00.703682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad776277-25ae-4547-a69b-bf4962f7c859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:00.704421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad776277-25ae-4547-a69b-bf4962f7c859 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:00.704805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad776277-25ae-4547-a69b-bf4962f7c859 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:00.705180 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad776277-25ae-4547-a69b-bf4962f7c859 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:00.705992 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad776277-25ae-4547-a69b-bf4962f7c859 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:00.706401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad776277-25ae-4547-a69b-bf4962f7c859 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:00.707908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad776277-25ae-4547-a69b-bf4962f7c859 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:00.726000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 316/607] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:00.727432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7691c-65ba-45b5-af2e-245037e6399a None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:00.727845 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7691c-65ba-45b5-af2e-245037e6399a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:00.728250 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7691c-65ba-45b5-af2e-245037e6399a None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:00.729856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 317/608] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:00.730643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7691c-65ba-45b5-af2e-245037e6399a None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:00.751501 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019806df-f197-459e-8398-4c08384d85bd None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:00.788984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c6ccf-f2a8-4f5c-8414-9a92af9c4a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:00.789643 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c6ccf-f2a8-4f5c-8414-9a92af9c4a72 None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:00.789919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c6ccf-f2a8-4f5c-8414-9a92af9c4a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:00.790193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c6ccf-f2a8-4f5c-8414-9a92af9c4a72 None admin] PATH_INFO: `/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/9d90a7ed44fd4a1fa94b9d44939df425/roles/cd65aca5e0074973a10adcd1c9a180dc` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:00.807898 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c6ccf-f2a8-4f5c-8414-9a92af9c4a72 None admin] RBAC: Authorizing `identity:create_grant(project_id=11372fe8994543b4a19773c0e0bee53e, user_id=9d90a7ed44fd4a1fa94b9d44939df425, role_id=cd65aca5e0074973a10adcd1c9a180dc)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:00.809375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c6ccf-f2a8-4f5c-8414-9a92af9c4a72 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:00.835093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:00.839802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7691c-65ba-45b5-af2e-245037e6399a None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:00.872952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bc902-6109-4a14-aa19-994c27fa0037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:00.873576 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bc902-6109-4a14-aa19-994c27fa0037 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:00.873861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bc902-6109-4a14-aa19-994c27fa0037 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:00.874131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bc902-6109-4a14-aa19-994c27fa0037 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:00.884117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bc902-6109-4a14-aa19-994c27fa0037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-525bc902-6109-4a14-aa19-994c27fa0037 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:00.888943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:00.890056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:00.890881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:00.890881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:00.890881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:00.890881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:00.890881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:00.890881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:00.890881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:00.890881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:00.890881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:00.890881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:00.890881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:00.890881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:00.890881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 318/609] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:06:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:00.891553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bc902-6109-4a14-aa19-994c27fa0037 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:00.906912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 292/610] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:06:00 2022] PUT /identity/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/9d90a7ed44fd4a1fa94b9d44939df425/roles/cd65aca5e0074973a10adcd1c9a180dc => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:06:00.906912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ae09a-d18e-47c5-8ab6-b1dfa6463c5a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:00.906912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ae09a-d18e-47c5-8ab6-b1dfa6463c5a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:00.906912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ae09a-d18e-47c5-8ab6-b1dfa6463c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:00.932236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-979ae09a-d18e-47c5-8ab6-b1dfa6463c5a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:00.933896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-979ae09a-d18e-47c5-8ab6-b1dfa6463c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:00.995452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:01.027208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bc902-6109-4a14-aa19-994c27fa0037 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:01.034722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 293/611] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:06:01.035679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ae09a-d18e-47c5-8ab6-b1dfa6463c5a None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:01.071977 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1749aab-dd22-4962-b942-4cc3a665ee6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:01.073063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1749aab-dd22-4962-b942-4cc3a665ee6a None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:01.073562 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1749aab-dd22-4962-b942-4cc3a665ee6a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:01.074032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1749aab-dd22-4962-b942-4cc3a665ee6a None admin] PATH_INFO: `/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/aa222b1c89dc4d2cb1686b4838c7908b/roles/cd65aca5e0074973a10adcd1c9a180dc` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:01.100462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1749aab-dd22-4962-b942-4cc3a665ee6a None admin] RBAC: Authorizing `identity:create_grant(project_id=11372fe8994543b4a19773c0e0bee53e, user_id=aa222b1c89dc4d2cb1686b4838c7908b, role_id=cd65aca5e0074973a10adcd1c9a180dc)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:01.102292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1749aab-dd22-4962-b942-4cc3a665ee6a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:01.153146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ae09a-d18e-47c5-8ab6-b1dfa6463c5a None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:01.188873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d71e2-3a51-4266-b351-552643db2b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:01.189581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d71e2-3a51-4266-b351-552643db2b74 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:01.189833 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d71e2-3a51-4266-b351-552643db2b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:01.190109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d71e2-3a51-4266-b351-552643db2b74 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:01.190802 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d71e2-3a51-4266-b351-552643db2b74 None admin] RBAC: Adding query filter params (name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:01.191084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d71e2-3a51-4266-b351-552643db2b74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:01.192542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d71e2-3a51-4266-b351-552643db2b74 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:01.201661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 294/612] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:06:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:01.207970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d71e2-3a51-4266-b351-552643db2b74 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:01.221079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 319/613] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:06:00 2022] PUT /identity/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/aa222b1c89dc4d2cb1686b4838c7908b/roles/cd65aca5e0074973a10adcd1c9a180dc => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:06:01.248198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:01.318380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d71e2-3a51-4266-b351-552643db2b74 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:01.356506 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6305589-217a-492d-9062-1bd6c8245249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:01.357164 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6305589-217a-492d-9062-1bd6c8245249 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:01.357443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6305589-217a-492d-9062-1bd6c8245249 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:01.357718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6305589-217a-492d-9062-1bd6c8245249 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:01.370115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6305589-217a-492d-9062-1bd6c8245249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:01.371597 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6305589-217a-492d-9062-1bd6c8245249 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-c6305589-217a-492d-9062-1bd6c8245249 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:01.387442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:01.388584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:01.389573 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:01.389573 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:01.389573 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:01.389573 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 295/614] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:01.395361 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1749aab-dd22-4962-b942-4cc3a665ee6a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:01.419661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:01.425076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6305589-217a-492d-9062-1bd6c8245249 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:01.523654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1749aab-dd22-4962-b942-4cc3a665ee6a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:01.528535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:01.540735 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6305589-217a-492d-9062-1bd6c8245249 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:01.564045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28752811-c3d4-497f-b9fa-400d5c0292a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:01.565168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28752811-c3d4-497f-b9fa-400d5c0292a9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:01.565693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28752811-c3d4-497f-b9fa-400d5c0292a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:01.566200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28752811-c3d4-497f-b9fa-400d5c0292a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:01.567067 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28752811-c3d4-497f-b9fa-400d5c0292a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:01.568456 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28752811-c3d4-497f-b9fa-400d5c0292a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:01.570097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28752811-c3d4-497f-b9fa-400d5c0292a9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:01.580213 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348059d-a6c1-4835-9999-59b2d8fd964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:01.580896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348059d-a6c1-4835-9999-59b2d8fd964c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:01.581113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348059d-a6c1-4835-9999-59b2d8fd964c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:01.581389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348059d-a6c1-4835-9999-59b2d8fd964c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:01.584791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 320/615] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:01.596339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28752811-c3d4-497f-b9fa-400d5c0292a9 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:01.596517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348059d-a6c1-4835-9999-59b2d8fd964c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-f348059d-a6c1-4835-9999-59b2d8fd964c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:01.600117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:01.601118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:01.601989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:01.601989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:01.601989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:01.601989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:01.601989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:01.601989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:01.601989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:01.601989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:01.601989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:01.601989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:01.601989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:01.601989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:01.606799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 296/616] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:06:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:01.620444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348059d-a6c1-4835-9999-59b2d8fd964c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:01.720188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:01.732341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348059d-a6c1-4835-9999-59b2d8fd964c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:01.782735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28752811-c3d4-497f-b9fa-400d5c0292a9 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:01.797231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10b773-d961-4260-8c2a-4c9557b52284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:01.798317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10b773-d961-4260-8c2a-4c9557b52284 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:01.799472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10b773-d961-4260-8c2a-4c9557b52284 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:01.801163 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10b773-d961-4260-8c2a-4c9557b52284 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:01.802085 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10b773-d961-4260-8c2a-4c9557b52284 None admin] RBAC: Adding query filter params (name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:01.802686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10b773-d961-4260-8c2a-4c9557b52284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:01.804456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10b773-d961-4260-8c2a-4c9557b52284 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:01.821910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 297/617] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:06:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:01.825775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10b773-d961-4260-8c2a-4c9557b52284 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:01.850164 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:01.857728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874c994-f02f-4752-a9c5-9dccf10b9294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:01.863358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874c994-f02f-4752-a9c5-9dccf10b9294 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:01.863358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874c994-f02f-4752-a9c5-9dccf10b9294 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:01.863358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874c994-f02f-4752-a9c5-9dccf10b9294 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:01.874879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874c994-f02f-4752-a9c5-9dccf10b9294 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:01.877048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874c994-f02f-4752-a9c5-9dccf10b9294 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-e874c994-f02f-4752-a9c5-9dccf10b9294 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:01.888834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:01.889815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:01.891058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:01.891058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:01.891058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:01.891058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:01.891058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:01.891058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:01.891058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:01.891058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:01.891058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:06:01.891058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:01.891058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 321/618] 10.222.0.85 () {56 vars in 1093 bytes} [Mon Aug 29 20:06:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:01.895772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874c994-f02f-4752-a9c5-9dccf10b9294 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:01.963422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10b773-d961-4260-8c2a-4c9557b52284 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:02.009829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c6b75-620a-40bb-a955-52b84fc710e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:02.010458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c6b75-620a-40bb-a955-52b84fc710e4 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:02.010736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c6b75-620a-40bb-a955-52b84fc710e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:02.011007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c6b75-620a-40bb-a955-52b84fc710e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:02.022175 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c6b75-620a-40bb-a955-52b84fc710e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:02.023626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c6b75-620a-40bb-a955-52b84fc710e4 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-162c6b75-620a-40bb-a955-52b84fc710e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:02.038429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:02.039402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:02.040301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:02.040301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:02.040301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:02.040301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 298/619] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:02.049103 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c6b75-620a-40bb-a955-52b84fc710e4 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:02.087340 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874c994-f02f-4752-a9c5-9dccf10b9294 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:02.120116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:02.146377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:02.158940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c6b75-620a-40bb-a955-52b84fc710e4 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:02.159119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc7b1d-1e9c-4f55-8c4d-391b73d6aad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:02.159975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc7b1d-1e9c-4f55-8c4d-391b73d6aad2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:02.160411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc7b1d-1e9c-4f55-8c4d-391b73d6aad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:02.160688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc7b1d-1e9c-4f55-8c4d-391b73d6aad2 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:02.161424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc7b1d-1e9c-4f55-8c4d-391b73d6aad2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:02.161998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc7b1d-1e9c-4f55-8c4d-391b73d6aad2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:02.164405 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc7b1d-1e9c-4f55-8c4d-391b73d6aad2 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:02.195252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9cf7d-1a72-42d9-a4f3-0d791870a824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:02.195922 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9cf7d-1a72-42d9-a4f3-0d791870a824 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:02.196178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9cf7d-1a72-42d9-a4f3-0d791870a824 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:02.196459 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9cf7d-1a72-42d9-a4f3-0d791870a824 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:02.197099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9cf7d-1a72-42d9-a4f3-0d791870a824 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:02.197401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9cf7d-1a72-42d9-a4f3-0d791870a824 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:02.198764 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9cf7d-1a72-42d9-a4f3-0d791870a824 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:02.203496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 322/620] 10.222.0.85 () {56 vars in 1133 bytes} [Mon Aug 29 20:06:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:02.209172 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc7b1d-1e9c-4f55-8c4d-391b73d6aad2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:02.212978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 299/621] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:02.218489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9cf7d-1a72-42d9-a4f3-0d791870a824 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:02.339024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9cf7d-1a72-42d9-a4f3-0d791870a824 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:02.363696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:02.377350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc7b1d-1e9c-4f55-8c4d-391b73d6aad2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:02.377859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4001b-e15c-4596-a475-6fca303b55ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:02.378233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4001b-e15c-4596-a475-6fca303b55ca None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:02.378515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4001b-e15c-4596-a475-6fca303b55ca None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:02.378813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4001b-e15c-4596-a475-6fca303b55ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:02.386170 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4001b-e15c-4596-a475-6fca303b55ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:02.387644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4001b-e15c-4596-a475-6fca303b55ca None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-b9d4001b-e15c-4596-a475-6fca303b55ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:02.401700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:02.403854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:02.406135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:02.406135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:02.406135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:02.406135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:02.406135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:02.406135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:02.406135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:02.406135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:02.406135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:06:02.406135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:02.406135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 300/622] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:06:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:02.406135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd34ad-cdc3-4549-b769-c50004d9d153 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:02.406135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd34ad-cdc3-4549-b769-c50004d9d153 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:02.407431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd34ad-cdc3-4549-b769-c50004d9d153 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:02.407431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 301/623] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:02.412219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd34ad-cdc3-4549-b769-c50004d9d153 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:02.419227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35544910-f93a-497b-a739-ea410773379f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:02.419841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35544910-f93a-497b-a739-ea410773379f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:02.420116 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35544910-f93a-497b-a739-ea410773379f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:02.420419 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35544910-f93a-497b-a739-ea410773379f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:02.488620 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:02.501161 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35544910-f93a-497b-a739-ea410773379f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:02.502791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35544910-f93a-497b-a739-ea410773379f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:02.523760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd34ad-cdc3-4549-b769-c50004d9d153 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:02.560821 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3d9c1-7961-480c-922c-7a54f5809d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:02.561647 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3d9c1-7961-480c-922c-7a54f5809d03 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:02.562179 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3d9c1-7961-480c-922c-7a54f5809d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:02.562575 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3d9c1-7961-480c-922c-7a54f5809d03 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:02.563534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3d9c1-7961-480c-922c-7a54f5809d03 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:02.563952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3d9c1-7961-480c-922c-7a54f5809d03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:02.565535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3d9c1-7961-480c-922c-7a54f5809d03 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:02.585532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 302/624] 10.222.0.85 () {56 vars in 1135 bytes} [Mon Aug 29 20:06:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:02.587348 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90f8f4-838e-48ca-ae9b-0802cca85f04 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:02.587863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90f8f4-838e-48ca-ae9b-0802cca85f04 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:02.588378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90f8f4-838e-48ca-ae9b-0802cca85f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:02.619620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-dd90f8f4-838e-48ca-ae9b-0802cca85f04 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:02.620855 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-dd90f8f4-838e-48ca-ae9b-0802cca85f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:02.638195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 323/625] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:06:02.639496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35544910-f93a-497b-a739-ea410773379f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:02.662512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:02.708213 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 303/626] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:06:02.708213 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90f8f4-838e-48ca-ae9b-0802cca85f04 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:02.758914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:02.776692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35544910-f93a-497b-a739-ea410773379f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:02.816422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fcecb-429c-477d-8acf-7955d539c61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:02.817062 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fcecb-429c-477d-8acf-7955d539c61d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:02.817340 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fcecb-429c-477d-8acf-7955d539c61d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:02.817610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fcecb-429c-477d-8acf-7955d539c61d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:02.841384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fcecb-429c-477d-8acf-7955d539c61d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:02.842873 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fcecb-429c-477d-8acf-7955d539c61d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-0f7fcecb-429c-477d-8acf-7955d539c61d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:02.865059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:02.866013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:02.866871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:02.866871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:02.866871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:02.867178 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 324/627] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:02.872442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fcecb-429c-477d-8acf-7955d539c61d None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:02.885040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90f8f4-838e-48ca-ae9b-0802cca85f04 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:02.950796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b94a0f-fea9-4546-bed2-90e9a7486b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:02.951431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b94a0f-fea9-4546-bed2-90e9a7486b3d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:02.951711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b94a0f-fea9-4546-bed2-90e9a7486b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:02.951984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b94a0f-fea9-4546-bed2-90e9a7486b3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:02.972351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b94a0f-fea9-4546-bed2-90e9a7486b3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:02.973858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b94a0f-fea9-4546-bed2-90e9a7486b3d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-e7b94a0f-fea9-4546-bed2-90e9a7486b3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:02.993582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:02.994657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:02.995532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:02.995532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:02.995532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:02.995532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 304/628] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:03.001082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b94a0f-fea9-4546-bed2-90e9a7486b3d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:03.028767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:03.038781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fcecb-429c-477d-8acf-7955d539c61d None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:03.074266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f1912-c27a-4f36-84e3-5f7a0acc5808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:03.075491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f1912-c27a-4f36-84e3-5f7a0acc5808 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.075996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f1912-c27a-4f36-84e3-5f7a0acc5808 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.076376 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f1912-c27a-4f36-84e3-5f7a0acc5808 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.077118 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f1912-c27a-4f36-84e3-5f7a0acc5808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:03.077514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f1912-c27a-4f36-84e3-5f7a0acc5808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:03.079051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f1912-c27a-4f36-84e3-5f7a0acc5808 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:03.086575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:03.096830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 325/629] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:03.096830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f1912-c27a-4f36-84e3-5f7a0acc5808 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:03.133692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b94a0f-fea9-4546-bed2-90e9a7486b3d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:03.177140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0caf754-0aea-4c22-8e60-89d67939c669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:03.177908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0caf754-0aea-4c22-8e60-89d67939c669 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.178281 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0caf754-0aea-4c22-8e60-89d67939c669 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.178646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0caf754-0aea-4c22-8e60-89d67939c669 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.179385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0caf754-0aea-4c22-8e60-89d67939c669 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:03.179783 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0caf754-0aea-4c22-8e60-89d67939c669 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:03.181274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0caf754-0aea-4c22-8e60-89d67939c669 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:03.194495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 305/630] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:03.195533 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0caf754-0aea-4c22-8e60-89d67939c669 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:03.205345 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f1912-c27a-4f36-84e3-5f7a0acc5808 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:03.244232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024e5c5-cb68-4f26-97eb-5276e8cc870a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:03.245024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024e5c5-cb68-4f26-97eb-5276e8cc870a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.245399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024e5c5-cb68-4f26-97eb-5276e8cc870a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.245770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024e5c5-cb68-4f26-97eb-5276e8cc870a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.256956 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024e5c5-cb68-4f26-97eb-5276e8cc870a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-4024e5c5-cb68-4f26-97eb-5276e8cc870a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:03.262052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:03.263192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:03.264230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:03.264230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:03.264230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:03.264230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:03.264230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:03.264230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:03.264230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:03.264230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:03.264230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:03.264230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:03.264230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:03.264230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:03.270675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 326/631] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:06:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:03.270675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024e5c5-cb68-4f26-97eb-5276e8cc870a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:03.318388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:03.331994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:03.338534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0caf754-0aea-4c22-8e60-89d67939c669 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:03.374437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024e5c5-cb68-4f26-97eb-5276e8cc870a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:03.378529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad1dca-5704-4806-9664-030782229ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:03.378529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad1dca-5704-4806-9664-030782229ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.378795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad1dca-5704-4806-9664-030782229ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.379741 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad1dca-5704-4806-9664-030782229ac7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.388240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad1dca-5704-4806-9664-030782229ac7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:03.389710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad1dca-5704-4806-9664-030782229ac7 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-78ad1dca-5704-4806-9664-030782229ac7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:03.402177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:03.403077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:03.404271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:03.404271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:03.404271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:03.404271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:03.404271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:03.404271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:03.404271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:03.404271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:03.404271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 306/632] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:03.404271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad1dca-5704-4806-9664-030782229ac7 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:03.423051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd22f81-e17a-4d1b-819e-44050289242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:03.423709 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd22f81-e17a-4d1b-819e-44050289242d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.423987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd22f81-e17a-4d1b-819e-44050289242d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.424307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd22f81-e17a-4d1b-819e-44050289242d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.431278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd22f81-e17a-4d1b-819e-44050289242d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:03.432740 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd22f81-e17a-4d1b-819e-44050289242d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-8dd22f81-e17a-4d1b-819e-44050289242d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:03.445298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:03.446262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:03.447155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:03.447155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:03.447155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:03.447155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:03.447155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:03.447155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:03.447155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:03.447155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:03.447155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 327/633] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:03.447633 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7a08e-3f75-4990-afeb-ced20973c6fb None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.448009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7a08e-3f75-4990-afeb-ced20973c6fb None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.448098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7a08e-3f75-4990-afeb-ced20973c6fb None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.453579 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 328/634] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:03.453579 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7a08e-3f75-4990-afeb-ced20973c6fb None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:03.521200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad1dca-5704-4806-9664-030782229ac7 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:03.553297 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7a08e-3f75-4990-afeb-ced20973c6fb None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:03.555716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caed367-6930-49c5-858e-92b9dc21e8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:03.556359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caed367-6930-49c5-858e-92b9dc21e8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.556645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caed367-6930-49c5-858e-92b9dc21e8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.556917 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caed367-6930-49c5-858e-92b9dc21e8f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.557585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caed367-6930-49c5-858e-92b9dc21e8f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:03.557885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caed367-6930-49c5-858e-92b9dc21e8f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:03.559250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caed367-6930-49c5-858e-92b9dc21e8f3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:03.567864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 307/635] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:06:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:03.568694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caed367-6930-49c5-858e-92b9dc21e8f3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:03.572893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:03.574342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:03.587815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570fe1c-6da1-41a3-b846-0fe83d9904f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:03.593557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570fe1c-6da1-41a3-b846-0fe83d9904f7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.593557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570fe1c-6da1-41a3-b846-0fe83d9904f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.593557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570fe1c-6da1-41a3-b846-0fe83d9904f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.593557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570fe1c-6da1-41a3-b846-0fe83d9904f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:03.593557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570fe1c-6da1-41a3-b846-0fe83d9904f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:03.593557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570fe1c-6da1-41a3-b846-0fe83d9904f7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:03.603214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 329/636] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:03.604645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca72f68-57b0-4195-bd0e-7b225314758a None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.604951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca72f68-57b0-4195-bd0e-7b225314758a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.605220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca72f68-57b0-4195-bd0e-7b225314758a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.629584 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-2ca72f68-57b0-4195-bd0e-7b225314758a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:03.630796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2ca72f68-57b0-4195-bd0e-7b225314758a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:03.676253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caed367-6930-49c5-858e-92b9dc21e8f3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:03.705261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 330/637] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:06:03.706512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5559962-6fbc-4a5e-ac97-fa01cdab3f06 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.706794 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5559962-6fbc-4a5e-ac97-fa01cdab3f06 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.707064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5559962-6fbc-4a5e-ac97-fa01cdab3f06 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.708302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 331/638] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:03.708979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5559962-6fbc-4a5e-ac97-fa01cdab3f06 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:03.723954 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24af3a6-c942-492a-9395-30701bfe6167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:03.724672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24af3a6-c942-492a-9395-30701bfe6167 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.724951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24af3a6-c942-492a-9395-30701bfe6167 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.725233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24af3a6-c942-492a-9395-30701bfe6167 None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.725930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24af3a6-c942-492a-9395-30701bfe6167 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:03.726251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24af3a6-c942-492a-9395-30701bfe6167 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:03.727660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24af3a6-c942-492a-9395-30701bfe6167 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:03.740009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 308/639] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:03.740826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24af3a6-c942-492a-9395-30701bfe6167 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:03.808976 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:03.824875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5559962-6fbc-4a5e-ac97-fa01cdab3f06 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:03.834538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:03.848040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24af3a6-c942-492a-9395-30701bfe6167 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:03.862489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd2bb0-e1b5-49cb-8075-f1451c49f748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:03.863120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd2bb0-e1b5-49cb-8075-f1451c49f748 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.863396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd2bb0-e1b5-49cb-8075-f1451c49f748 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.868157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd2bb0-e1b5-49cb-8075-f1451c49f748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.873972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd2bb0-e1b5-49cb-8075-f1451c49f748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:03.875483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd2bb0-e1b5-49cb-8075-f1451c49f748 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:03.887497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aae336-675d-4fe4-8bcb-424ef604de09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:03.888259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aae336-675d-4fe4-8bcb-424ef604de09 None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.888570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aae336-675d-4fe4-8bcb-424ef604de09 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.888866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aae336-675d-4fe4-8bcb-424ef604de09 None admin] PATH_INFO: `/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/d52f84245a2142e69c309aa709a7d921/roles/cd65aca5e0074973a10adcd1c9a180dc` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-35fd2bb0-e1b5-49cb-8075-f1451c49f748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:03.891592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:03.892612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:03.893529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:03.893529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:03.893529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:03.898751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 332/640] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:03.900015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323c59f-55e8-42ad-a26e-d8943f263310 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:03.900543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323c59f-55e8-42ad-a26e-d8943f263310 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:03.900799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323c59f-55e8-42ad-a26e-d8943f263310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:03.907455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aae336-675d-4fe4-8bcb-424ef604de09 None admin] RBAC: Authorizing `identity:create_grant(project_id=11372fe8994543b4a19773c0e0bee53e, user_id=d52f84245a2142e69c309aa709a7d921, role_id=cd65aca5e0074973a10adcd1c9a180dc)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:03.909013 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aae336-675d-4fe4-8bcb-424ef604de09 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:03.920139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-2323c59f-55e8-42ad-a26e-d8943f263310 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:03.921406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2323c59f-55e8-42ad-a26e-d8943f263310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:03.989460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 333/641] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:06:03.990373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323c59f-55e8-42ad-a26e-d8943f263310 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:04.025413 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 309/642] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:06:03 2022] PUT /identity/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/d52f84245a2142e69c309aa709a7d921/roles/cd65aca5e0074973a10adcd1c9a180dc => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:06:04.026528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aae336-675d-4fe4-8bcb-424ef604de09 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:04.104592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:04.117955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323c59f-55e8-42ad-a26e-d8943f263310 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:04.163886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aae336-675d-4fe4-8bcb-424ef604de09 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:04.166070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe95cba-19eb-4ce7-9ade-57af74fc6796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:04.166773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe95cba-19eb-4ce7-9ade-57af74fc6796 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:04.167049 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe95cba-19eb-4ce7-9ade-57af74fc6796 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:04.167323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe95cba-19eb-4ce7-9ade-57af74fc6796 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:04.172366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe95cba-19eb-4ce7-9ade-57af74fc6796 None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=805df3047fee45c7a702d72a5c082eb5) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:04.172759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe95cba-19eb-4ce7-9ade-57af74fc6796 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:04.174889 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe95cba-19eb-4ce7-9ade-57af74fc6796 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:04.188219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 334/643] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:03 2022] GET /identity/v3/role_assignments?user.id=805df3047fee45c7a702d72a5c082eb5&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:04.188410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:04.188515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe95cba-19eb-4ce7-9ade-57af74fc6796 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:04.216293 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf494635-d6e1-4c5b-afc9-1d0514057ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:04.216961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf494635-d6e1-4c5b-afc9-1d0514057ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:04.217314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf494635-d6e1-4c5b-afc9-1d0514057ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:04.217589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf494635-d6e1-4c5b-afc9-1d0514057ce6 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:04.219527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf494635-d6e1-4c5b-afc9-1d0514057ce6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:04.219845 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf494635-d6e1-4c5b-afc9-1d0514057ce6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:04.221601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf494635-d6e1-4c5b-afc9-1d0514057ce6 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:04.251491 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 310/644] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:04.251491 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf494635-d6e1-4c5b-afc9-1d0514057ce6 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:04.325397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe95cba-19eb-4ce7-9ade-57af74fc6796 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:04.370918 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc84387-9291-4b35-9ab6-09d1b211fbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:04.371593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc84387-9291-4b35-9ab6-09d1b211fbc9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:04.371908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc84387-9291-4b35-9ab6-09d1b211fbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:04.372197 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc84387-9291-4b35-9ab6-09d1b211fbc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:04.384797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc84387-9291-4b35-9ab6-09d1b211fbc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:04.389976 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf494635-d6e1-4c5b-afc9-1d0514057ce6 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-cdc84387-9291-4b35-9ab6-09d1b211fbc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:04.390128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:04.391564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:04.392475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:04.392475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:04.392475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:04.392475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:04.392475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:04.392475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:04.392475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:04.392475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:04.392475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:04.392475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:04.392475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:04.392475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:04.392475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 335/645] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:06:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:04.397019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc84387-9291-4b35-9ab6-09d1b211fbc9 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:04.417316 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:04.430063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a134af-9d93-4752-90ef-20c972e1ac8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:04.431052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a134af-9d93-4752-90ef-20c972e1ac8e None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:04.431335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a134af-9d93-4752-90ef-20c972e1ac8e None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:04.431607 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a134af-9d93-4752-90ef-20c972e1ac8e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:04.440956 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a134af-9d93-4752-90ef-20c972e1ac8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:04.442515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a134af-9d93-4752-90ef-20c972e1ac8e None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-32a134af-9d93-4752-90ef-20c972e1ac8e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:04.459277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:04.460477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:04.461560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:04.461560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:04.461560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:04.461560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:04.461560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:04.461560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:04.461560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:04.461560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:04.461560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:06:04.461560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:04.467162 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 311/646] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:06:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:04.467162 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a134af-9d93-4752-90ef-20c972e1ac8e None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:04.503798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:04.535251 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc84387-9291-4b35-9ab6-09d1b211fbc9 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:04.577879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a134af-9d93-4752-90ef-20c972e1ac8e None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:04.579671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fce3a6-5620-4dab-922b-1ec3dfa66cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:04.581272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fce3a6-5620-4dab-922b-1ec3dfa66cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:04.581643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fce3a6-5620-4dab-922b-1ec3dfa66cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:04.581976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fce3a6-5620-4dab-922b-1ec3dfa66cf5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:04.597286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fce3a6-5620-4dab-922b-1ec3dfa66cf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-39fce3a6-5620-4dab-922b-1ec3dfa66cf5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:04.607548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:04.608551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 336/647] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:06:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:04.609957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fce3a6-5620-4dab-922b-1ec3dfa66cf5 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:04.627331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046d02d-2ada-4fed-ad33-a780a5b53069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:04.628004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046d02d-2ada-4fed-ad33-a780a5b53069 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:04.628321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046d02d-2ada-4fed-ad33-a780a5b53069 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:04.628638 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046d02d-2ada-4fed-ad33-a780a5b53069 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:04.629316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046d02d-2ada-4fed-ad33-a780a5b53069 None admin] RBAC: Adding query filter params (name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:04.629618 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046d02d-2ada-4fed-ad33-a780a5b53069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:04.631045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046d02d-2ada-4fed-ad33-a780a5b53069 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:04.644479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 312/648] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:06:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:04.645385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046d02d-2ada-4fed-ad33-a780a5b53069 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:04.720550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:04.746148 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fce3a6-5620-4dab-922b-1ec3dfa66cf5 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:04.761023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:04.766988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046d02d-2ada-4fed-ad33-a780a5b53069 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:04.794410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20ba73-7605-4205-b219-5aa3b8aca76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:04.795573 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20ba73-7605-4205-b219-5aa3b8aca76a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:04.796736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20ba73-7605-4205-b219-5aa3b8aca76a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:04.798250 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20ba73-7605-4205-b219-5aa3b8aca76a None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:04.799587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20ba73-7605-4205-b219-5aa3b8aca76a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:04.800751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20ba73-7605-4205-b219-5aa3b8aca76a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:04.802683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20ba73-7605-4205-b219-5aa3b8aca76a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:04.805111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad8d32-5f0a-4a8a-bf49-f198fb9b57cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:04.805796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad8d32-5f0a-4a8a-bf49-f198fb9b57cb None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:04.806078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad8d32-5f0a-4a8a-bf49-f198fb9b57cb None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:04.806350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad8d32-5f0a-4a8a-bf49-f198fb9b57cb None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:04.807434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad8d32-5f0a-4a8a-bf49-f198fb9b57cb None admin] RBAC: Adding query filter params (name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:04.807434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad8d32-5f0a-4a8a-bf49-f198fb9b57cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:04.808740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad8d32-5f0a-4a8a-bf49-f198fb9b57cb None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:04.823080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 313/649] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:06:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:04.823708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad8d32-5f0a-4a8a-bf49-f198fb9b57cb None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:04.828517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 337/650] 10.222.0.85 () {56 vars in 1137 bytes} [Mon Aug 29 20:06:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:04.833897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20ba73-7605-4205-b219-5aa3b8aca76a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:04.962354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20ba73-7605-4205-b219-5aa3b8aca76a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:04.966316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad8d32-5f0a-4a8a-bf49-f198fb9b57cb None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:04.996686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:05.002122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84dda3-65f8-4819-90b7-66f492e7984b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:05.002734 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84dda3-65f8-4819-90b7-66f492e7984b None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:05.003010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84dda3-65f8-4819-90b7-66f492e7984b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:05.003279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84dda3-65f8-4819-90b7-66f492e7984b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:05.017946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f63fa6-1ca6-413f-8f66-a7af9e68d609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:05.018982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f63fa6-1ca6-413f-8f66-a7af9e68d609 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:05.019420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f63fa6-1ca6-413f-8f66-a7af9e68d609 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:05.019848 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f63fa6-1ca6-413f-8f66-a7af9e68d609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:05.038663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f63fa6-1ca6-413f-8f66-a7af9e68d609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:05.041713 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f63fa6-1ca6-413f-8f66-a7af9e68d609 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-05f63fa6-1ca6-413f-8f66-a7af9e68d609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:05.067915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:05.069142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:05.070234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:05.070234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:05.070234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:05.072563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 314/651] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:05.073832 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f63fa6-1ca6-413f-8f66-a7af9e68d609 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:05.089231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:05.097543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84dda3-65f8-4819-90b7-66f492e7984b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:05.102986 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84dda3-65f8-4819-90b7-66f492e7984b None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:05.202300 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f63fa6-1ca6-413f-8f66-a7af9e68d609 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:05.224367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 338/652] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:06:05.225550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84dda3-65f8-4819-90b7-66f492e7984b None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:05.239642 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b249d17-e88d-4e68-b833-4e507017aa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:05.241724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b249d17-e88d-4e68-b833-4e507017aa51 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:05.242527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b249d17-e88d-4e68-b833-4e507017aa51 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:05.243286 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b249d17-e88d-4e68-b833-4e507017aa51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:05.257033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b249d17-e88d-4e68-b833-4e507017aa51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:05.258948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b249d17-e88d-4e68-b833-4e507017aa51 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:05.261727 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-6b249d17-e88d-4e68-b833-4e507017aa51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:05.283661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:05.284687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:05.285672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:05.285672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:05.285672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:05.285672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 315/653] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:05.285672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b249d17-e88d-4e68-b833-4e507017aa51 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:05.359833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84dda3-65f8-4819-90b7-66f492e7984b None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:05.362767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:05.408289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2403d7f-062a-4dac-9276-2dc8b190fab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:05.409090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2403d7f-062a-4dac-9276-2dc8b190fab5 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:05.409090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2403d7f-062a-4dac-9276-2dc8b190fab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:05.409289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2403d7f-062a-4dac-9276-2dc8b190fab5 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:05.410448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2403d7f-062a-4dac-9276-2dc8b190fab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:05.410448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2403d7f-062a-4dac-9276-2dc8b190fab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:05.411454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2403d7f-062a-4dac-9276-2dc8b190fab5 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:05.417361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b249d17-e88d-4e68-b833-4e507017aa51 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:05.426659 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 339/654] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:05.427519 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2403d7f-062a-4dac-9276-2dc8b190fab5 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:05.474790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bed838-160b-4129-bc7c-1bb4c9afe4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:05.475828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bed838-160b-4129-bc7c-1bb4c9afe4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:05.476434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bed838-160b-4129-bc7c-1bb4c9afe4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:05.476918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bed838-160b-4129-bc7c-1bb4c9afe4a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:05.491888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bed838-160b-4129-bc7c-1bb4c9afe4a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:05.494653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bed838-160b-4129-bc7c-1bb4c9afe4a4 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-02bed838-160b-4129-bc7c-1bb4c9afe4a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:05.513951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:05.514960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:05.515890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:05.515890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:05.515890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:05.519708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 316/655] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:05.520663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bed838-160b-4129-bc7c-1bb4c9afe4a4 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:05.570913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:05.581573 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2403d7f-062a-4dac-9276-2dc8b190fab5 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:05.637760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fa9e5-5bba-4566-b11c-58ac54ddc07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:05.638452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fa9e5-5bba-4566-b11c-58ac54ddc07c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:05.638740 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fa9e5-5bba-4566-b11c-58ac54ddc07c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:05.639097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fa9e5-5bba-4566-b11c-58ac54ddc07c None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:05.639697 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fa9e5-5bba-4566-b11c-58ac54ddc07c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:05.640002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fa9e5-5bba-4566-b11c-58ac54ddc07c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:05.641978 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fa9e5-5bba-4566-b11c-58ac54ddc07c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:05.657112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 340/656] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:05.658401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fa9e5-5bba-4566-b11c-58ac54ddc07c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:05.661313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bed838-160b-4129-bc7c-1bb4c9afe4a4 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:05.680936 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:05.704806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1275e-c7ae-4e8c-8d43-479417c3e5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:05.705442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1275e-c7ae-4e8c-8d43-479417c3e5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:05.705722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1275e-c7ae-4e8c-8d43-479417c3e5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:05.705994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1275e-c7ae-4e8c-8d43-479417c3e5d3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:05.714851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e1275e-c7ae-4e8c-8d43-479417c3e5d3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:05.716487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e1275e-c7ae-4e8c-8d43-479417c3e5d3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-94e1275e-c7ae-4e8c-8d43-479417c3e5d3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:05.730446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:05.731765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:05.733393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:05.733393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:05.733393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:05.733393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:05.733393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:05.733393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:05.733393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:05.733393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:05.733393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:06:05.733393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:05.733944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 317/657] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:06:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:05.734691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1275e-c7ae-4e8c-8d43-479417c3e5d3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:05.784355 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fa9e5-5bba-4566-b11c-58ac54ddc07c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:05.821056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c09f0-cf23-469c-846d-802e7a537d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:05.822045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c09f0-cf23-469c-846d-802e7a537d7d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:05.822512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c09f0-cf23-469c-846d-802e7a537d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:05.822980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c09f0-cf23-469c-846d-802e7a537d7d None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:05.823825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c09f0-cf23-469c-846d-802e7a537d7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:05.824388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c09f0-cf23-469c-846d-802e7a537d7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:05.826055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c09f0-cf23-469c-846d-802e7a537d7d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:05.839512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 341/658] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:05.840625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c09f0-cf23-469c-846d-802e7a537d7d None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:05.849939 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1275e-c7ae-4e8c-8d43-479417c3e5d3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:05.859066 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:05.887856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1c88c-94c9-4732-828e-e95d3a02079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:05.888794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1c88c-94c9-4732-828e-e95d3a02079b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:05.888794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1c88c-94c9-4732-828e-e95d3a02079b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:05.889075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1c88c-94c9-4732-828e-e95d3a02079b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:05.898303 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1c88c-94c9-4732-828e-e95d3a02079b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:05.899749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1c88c-94c9-4732-828e-e95d3a02079b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-3dc1c88c-94c9-4732-828e-e95d3a02079b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:05.915805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:05.916871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:05.917819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:05.917819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:05.917819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:05.917819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:05.917819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:05.917819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:05.917819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:05.917819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:05.917819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:06:05.917819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:05.917819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 318/659] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:06:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:05.918415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1c88c-94c9-4732-828e-e95d3a02079b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:05.956859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:05.976857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c09f0-cf23-469c-846d-802e7a537d7d None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:06.017606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335b71c-f1fb-4bba-b2a5-4088fbd8078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:06.018534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335b71c-f1fb-4bba-b2a5-4088fbd8078f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:06.019000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335b71c-f1fb-4bba-b2a5-4088fbd8078f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:06.019486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335b71c-f1fb-4bba-b2a5-4088fbd8078f None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:06.020533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335b71c-f1fb-4bba-b2a5-4088fbd8078f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:06.021025 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335b71c-f1fb-4bba-b2a5-4088fbd8078f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:06.022656 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335b71c-f1fb-4bba-b2a5-4088fbd8078f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:06.033819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1c88c-94c9-4732-828e-e95d3a02079b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:06.043921 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 342/660] 10.222.0.85 () {56 vars in 1137 bytes} [Mon Aug 29 20:06:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:06.045252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335b71c-f1fb-4bba-b2a5-4088fbd8078f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:06.071336 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07d9d3-a722-4596-a38c-ff88f1ab52e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:06.071911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07d9d3-a722-4596-a38c-ff88f1ab52e6 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:06.072229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07d9d3-a722-4596-a38c-ff88f1ab52e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:06.072510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07d9d3-a722-4596-a38c-ff88f1ab52e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:06.086580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07d9d3-a722-4596-a38c-ff88f1ab52e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:06.088044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07d9d3-a722-4596-a38c-ff88f1ab52e6 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-ce07d9d3-a722-4596-a38c-ff88f1ab52e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:06.105446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:06.106475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:06.107595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:06.107595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:06.107595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:06.107595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:06.107595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:06.107595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:06.107595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:06.107595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:06.108582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 319/661] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:06.109131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07d9d3-a722-4596-a38c-ff88f1ab52e6 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:06.137373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:06.180515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335b71c-f1fb-4bba-b2a5-4088fbd8078f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:06.216460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27922488-d03b-4729-be1a-4b038a9a2a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:06.216824 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27922488-d03b-4729-be1a-4b038a9a2a1a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:06.217105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27922488-d03b-4729-be1a-4b038a9a2a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:06.217381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27922488-d03b-4729-be1a-4b038a9a2a1a None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:06.218115 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27922488-d03b-4729-be1a-4b038a9a2a1a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:06.219131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:06.219826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27922488-d03b-4729-be1a-4b038a9a2a1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:06.221208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27922488-d03b-4729-be1a-4b038a9a2a1a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:06.226636 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07d9d3-a722-4596-a38c-ff88f1ab52e6 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:06.244462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 343/662] 10.222.0.85 () {56 vars in 1139 bytes} [Mon Aug 29 20:06:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:06.245344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27922488-d03b-4729-be1a-4b038a9a2a1a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:06.262666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fc5bc-b1b8-4e35-a4ea-f5d1b7ec82b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:06.263587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fc5bc-b1b8-4e35-a4ea-f5d1b7ec82b0 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:06.263871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fc5bc-b1b8-4e35-a4ea-f5d1b7ec82b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:06.264465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fc5bc-b1b8-4e35-a4ea-f5d1b7ec82b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:06.338404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6fc5bc-b1b8-4e35-a4ea-f5d1b7ec82b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:06.339768 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6fc5bc-b1b8-4e35-a4ea-f5d1b7ec82b0 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:06.350363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27922488-d03b-4729-be1a-4b038a9a2a1a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:06.379050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:06.387401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd36d7-de88-499e-bea3-510e989d557d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:06.391574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd36d7-de88-499e-bea3-510e989d557d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:06.391574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd36d7-de88-499e-bea3-510e989d557d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:06.391574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd36d7-de88-499e-bea3-510e989d557d None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:06.391574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd36d7-de88-499e-bea3-510e989d557d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:06.391574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd36d7-de88-499e-bea3-510e989d557d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:06.391574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd36d7-de88-499e-bea3-510e989d557d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:06.408013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 344/663] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:06.408013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866cf12-846d-4d78-ae3c-07b7520b354b None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:06.408013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866cf12-846d-4d78-ae3c-07b7520b354b None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:06.408013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866cf12-846d-4d78-ae3c-07b7520b354b None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:06.408013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 345/664] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:06.408013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866cf12-846d-4d78-ae3c-07b7520b354b None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:06.447825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:06.461300 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 320/665] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:06:06.461300 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fc5bc-b1b8-4e35-a4ea-f5d1b7ec82b0 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:06.522094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866cf12-846d-4d78-ae3c-07b7520b354b None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:06.555481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547376fb-7c3a-4675-b462-41dbd5231852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:06.556081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547376fb-7c3a-4675-b462-41dbd5231852 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:06.556383 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547376fb-7c3a-4675-b462-41dbd5231852 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:06.556655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547376fb-7c3a-4675-b462-41dbd5231852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:06.559157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fc5bc-b1b8-4e35-a4ea-f5d1b7ec82b0 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:06.592796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10bbd6-fe23-453f-93bb-6d53052917ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:06.593319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10bbd6-fe23-453f-93bb-6d53052917ba None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:06.594177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10bbd6-fe23-453f-93bb-6d53052917ba None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:06.594494 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10bbd6-fe23-453f-93bb-6d53052917ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:06.595219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10bbd6-fe23-453f-93bb-6d53052917ba None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=a7e768fdd1054e40b6b4910e39a6e533) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:06.595531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10bbd6-fe23-453f-93bb-6d53052917ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:06.596990 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10bbd6-fe23-453f-93bb-6d53052917ba None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:06.604337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 321/666] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:06 2022] GET /identity/v3/role_assignments?user.id=a7e768fdd1054e40b6b4910e39a6e533&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:06.604337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cd312-4225-4a81-9d42-fed465d20a84 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:06.604337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cd312-4225-4a81-9d42-fed465d20a84 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:06.604665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cd312-4225-4a81-9d42-fed465d20a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:06.623074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:06.632724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-8d9cd312-4225-4a81-9d42-fed465d20a84 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:06.633951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8d9cd312-4225-4a81-9d42-fed465d20a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:06.638610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547376fb-7c3a-4675-b462-41dbd5231852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:06.640821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547376fb-7c3a-4675-b462-41dbd5231852 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:06.696384 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:06.724196 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 322/667] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:06:06.724196 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cd312-4225-4a81-9d42-fed465d20a84 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:06.774061 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 346/668] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:06:06.775635 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe257415-8aac-4089-bdf7-f797bd32be6f None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:06.775920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe257415-8aac-4089-bdf7-f797bd32be6f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:06.776904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe257415-8aac-4089-bdf7-f797bd32be6f None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:06.780214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 347/669] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:06.780214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe257415-8aac-4089-bdf7-f797bd32be6f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:06.846084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cd312-4225-4a81-9d42-fed465d20a84 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:06.898502 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:06.900764 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b5da6-f14f-4cab-8378-92647251b144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:06.901671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b5da6-f14f-4cab-8378-92647251b144 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:06.901958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b5da6-f14f-4cab-8378-92647251b144 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:06.902227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b5da6-f14f-4cab-8378-92647251b144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:06.914673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe257415-8aac-4089-bdf7-f797bd32be6f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:06.915941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b5da6-f14f-4cab-8378-92647251b144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:06.918463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b5da6-f14f-4cab-8378-92647251b144 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-8b1b5da6-f14f-4cab-8378-92647251b144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:06.938532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:06.939531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:06.940919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:06.940919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:06.940919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:06.940919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 323/670] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:06.941290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955bccf-1072-4893-bae0-1a9a9387e5bb None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:06.941516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955bccf-1072-4893-bae0-1a9a9387e5bb None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:06.941774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955bccf-1072-4893-bae0-1a9a9387e5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:06.956948 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b1eb5-4d35-460e-8c7b-529895998d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:06.957960 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b1eb5-4d35-460e-8c7b-529895998d9f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:06.958619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b1eb5-4d35-460e-8c7b-529895998d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:06.959237 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b1eb5-4d35-460e-8c7b-529895998d9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:06.965550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-7955bccf-1072-4893-bae0-1a9a9387e5bb None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:06.966842 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7955bccf-1072-4893-bae0-1a9a9387e5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:06.972991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b1eb5-4d35-460e-8c7b-529895998d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:06.974836 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b1eb5-4d35-460e-8c7b-529895998d9f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-b32b1eb5-4d35-460e-8c7b-529895998d9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:06.992868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:06.998975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:06.999946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:06.999946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:06.999946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:06.999946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 348/671] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:06.999946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b1eb5-4d35-460e-8c7b-529895998d9f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:07.009012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:07.046243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 324/672] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:06:07.047708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955bccf-1072-4893-bae0-1a9a9387e5bb None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:07.121537 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b1eb5-4d35-460e-8c7b-529895998d9f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:07.158021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72df63c-8d45-46e4-8926-dbb79be72e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:07.158638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72df63c-8d45-46e4-8926-dbb79be72e04 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:07.158926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72df63c-8d45-46e4-8926-dbb79be72e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:07.159444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72df63c-8d45-46e4-8926-dbb79be72e04 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:07.160114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72df63c-8d45-46e4-8926-dbb79be72e04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:07.160867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72df63c-8d45-46e4-8926-dbb79be72e04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:07.162546 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72df63c-8d45-46e4-8926-dbb79be72e04 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:07.166900 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955bccf-1072-4893-bae0-1a9a9387e5bb None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:07.175020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:07.180568 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 349/673] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:07.181395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72df63c-8d45-46e4-8926-dbb79be72e04 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:07.205306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3c5ed-8504-4fb2-8f2f-968c7036b945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:07.206343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3c5ed-8504-4fb2-8f2f-968c7036b945 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:07.206852 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3c5ed-8504-4fb2-8f2f-968c7036b945 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:07.207374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3c5ed-8504-4fb2-8f2f-968c7036b945 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:07.218277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3c5ed-8504-4fb2-8f2f-968c7036b945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-10a3c5ed-8504-4fb2-8f2f-968c7036b945 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:07.224230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:07.225246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:07.226071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:07.226071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:07.226071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:07.226071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:07.226071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:07.226071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:07.226071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:07.226071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:07.226071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:07.226071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:07.226071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:07.226071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:07.228748 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 325/674] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:06:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:07.229952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3c5ed-8504-4fb2-8f2f-968c7036b945 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:07.300648 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:07.308763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72df63c-8d45-46e4-8926-dbb79be72e04 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:07.339257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3c5ed-8504-4fb2-8f2f-968c7036b945 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:07.347215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba528e-1704-4fea-91dd-e5b740a13ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:07.347948 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba528e-1704-4fea-91dd-e5b740a13ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:07.348262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba528e-1704-4fea-91dd-e5b740a13ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:07.348539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba528e-1704-4fea-91dd-e5b740a13ba5 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:07.349189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba528e-1704-4fea-91dd-e5b740a13ba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:07.349495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba528e-1704-4fea-91dd-e5b740a13ba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:07.350996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba528e-1704-4fea-91dd-e5b740a13ba5 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:07.364572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 350/675] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:07.365533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba528e-1704-4fea-91dd-e5b740a13ba5 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:07.380016 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e56491-6b41-431e-a0e5-274f8aa04710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:07.380913 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e56491-6b41-431e-a0e5-274f8aa04710 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:07.381193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e56491-6b41-431e-a0e5-274f8aa04710 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:07.381463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e56491-6b41-431e-a0e5-274f8aa04710 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:07.388368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e56491-6b41-431e-a0e5-274f8aa04710 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:07.389809 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e56491-6b41-431e-a0e5-274f8aa04710 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-16e56491-6b41-431e-a0e5-274f8aa04710 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:07.402287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:07.403169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:07.404081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:07.404081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:07.404081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:07.404081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:07.404081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:07.404081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:07.404081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:07.404081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:07.404081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 326/676] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:07.404612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e56491-6b41-431e-a0e5-274f8aa04710 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:07.435177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:07.478023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba528e-1704-4fea-91dd-e5b740a13ba5 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:07.509259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:07.514777 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa68205-95ec-4f2e-8a1c-53b46f7dfd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:07.516225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa68205-95ec-4f2e-8a1c-53b46f7dfd44 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:07.516725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa68205-95ec-4f2e-8a1c-53b46f7dfd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:07.517100 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa68205-95ec-4f2e-8a1c-53b46f7dfd44 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:07.517766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa68205-95ec-4f2e-8a1c-53b46f7dfd44 None admin] RBAC: Adding query filter params (name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:07.518122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa68205-95ec-4f2e-8a1c-53b46f7dfd44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:07.525278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e56491-6b41-431e-a0e5-274f8aa04710 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:07.525422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa68205-95ec-4f2e-8a1c-53b46f7dfd44 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:07.528341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 351/677] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:06:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:07.529130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa68205-95ec-4f2e-8a1c-53b46f7dfd44 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:07.561332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810da31a-28dc-4c4a-973d-c2c5b2e1d811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:07.561971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810da31a-28dc-4c4a-973d-c2c5b2e1d811 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:07.562785 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810da31a-28dc-4c4a-973d-c2c5b2e1d811 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:07.563063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810da31a-28dc-4c4a-973d-c2c5b2e1d811 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:07.575059 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810da31a-28dc-4c4a-973d-c2c5b2e1d811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-810da31a-28dc-4c4a-973d-c2c5b2e1d811 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:07.580195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:07.581207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 327/678] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:06:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:07.582074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810da31a-28dc-4c4a-973d-c2c5b2e1d811 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:07.647818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa68205-95ec-4f2e-8a1c-53b46f7dfd44 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:07.674289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:07.686734 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c02e2-cf1d-4345-bd03-5c66f4d667e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:07.687557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c02e2-cf1d-4345-bd03-5c66f4d667e1 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:07.687843 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c02e2-cf1d-4345-bd03-5c66f4d667e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:07.688119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c02e2-cf1d-4345-bd03-5c66f4d667e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:07.695746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c02e2-cf1d-4345-bd03-5c66f4d667e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:07.696416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810da31a-28dc-4c4a-973d-c2c5b2e1d811 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:07.697281 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c02e2-cf1d-4345-bd03-5c66f4d667e1 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-d01c02e2-cf1d-4345-bd03-5c66f4d667e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:07.711094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:07.713068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:07.714892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:07.714892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:07.714892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:07.714892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:07.714892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:07.714892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:07.714892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:07.714892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:07.714892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 352/679] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:07.714892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c02e2-cf1d-4345-bd03-5c66f4d667e1 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:07.742574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56760cc7-24d3-49a3-b1f4-694b579bc4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:07.743186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56760cc7-24d3-49a3-b1f4-694b579bc4df None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:07.743464 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56760cc7-24d3-49a3-b1f4-694b579bc4df None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:07.743735 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56760cc7-24d3-49a3-b1f4-694b579bc4df None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:07.744456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56760cc7-24d3-49a3-b1f4-694b579bc4df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:07.744761 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56760cc7-24d3-49a3-b1f4-694b579bc4df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:07.746187 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56760cc7-24d3-49a3-b1f4-694b579bc4df None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:07.758355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 328/680] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:07.759020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56760cc7-24d3-49a3-b1f4-694b579bc4df None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:07.774090 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:07.826075 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c02e2-cf1d-4345-bd03-5c66f4d667e1 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:07.862156 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282ac43-181f-45a3-92a6-f936390a7871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:07.862791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282ac43-181f-45a3-92a6-f936390a7871 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:07.863070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282ac43-181f-45a3-92a6-f936390a7871 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:07.863344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282ac43-181f-45a3-92a6-f936390a7871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:07.871274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56760cc7-24d3-49a3-b1f4-694b579bc4df None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:07.875105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282ac43-181f-45a3-92a6-f936390a7871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:07.876601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282ac43-181f-45a3-92a6-f936390a7871 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-e282ac43-181f-45a3-92a6-f936390a7871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:07.893676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:07.894780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:07.895676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:07.895676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:07.895676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:07.895676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 353/681] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:07.895676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282ac43-181f-45a3-92a6-f936390a7871 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:07.915232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f09e26-dfdb-4a0b-8e8a-163f23645563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:07.915851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f09e26-dfdb-4a0b-8e8a-163f23645563 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:07.916134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f09e26-dfdb-4a0b-8e8a-163f23645563 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:07.916450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f09e26-dfdb-4a0b-8e8a-163f23645563 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:07.917126 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f09e26-dfdb-4a0b-8e8a-163f23645563 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:07.917428 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f09e26-dfdb-4a0b-8e8a-163f23645563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:07.918792 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f09e26-dfdb-4a0b-8e8a-163f23645563 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:07.927348 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 329/682] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:06:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:07.927964 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f09e26-dfdb-4a0b-8e8a-163f23645563 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:07.941346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:08.006081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:08.012674 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282ac43-181f-45a3-92a6-f936390a7871 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:08.039680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f09e26-dfdb-4a0b-8e8a-163f23645563 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:08.047904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f60b89-0111-4b92-8d1e-a1df6d368f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:08.048594 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f60b89-0111-4b92-8d1e-a1df6d368f55 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:08.048880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f60b89-0111-4b92-8d1e-a1df6d368f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:08.049153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f60b89-0111-4b92-8d1e-a1df6d368f55 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:08.049845 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f60b89-0111-4b92-8d1e-a1df6d368f55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:08.050341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f60b89-0111-4b92-8d1e-a1df6d368f55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:08.051593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f60b89-0111-4b92-8d1e-a1df6d368f55 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:08.069176 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 354/683] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:08.069838 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f60b89-0111-4b92-8d1e-a1df6d368f55 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:08.077187 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bdef6-66ce-4c16-b637-42cbfbdf8511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:08.078320 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bdef6-66ce-4c16-b637-42cbfbdf8511 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:08.078641 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bdef6-66ce-4c16-b637-42cbfbdf8511 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:08.078919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bdef6-66ce-4c16-b637-42cbfbdf8511 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:08.079646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bdef6-66ce-4c16-b637-42cbfbdf8511 None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=9d90a7ed44fd4a1fa94b9d44939df425) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:08.079959 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bdef6-66ce-4c16-b637-42cbfbdf8511 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:08.081427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bdef6-66ce-4c16-b637-42cbfbdf8511 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:08.087549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 330/684] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:07 2022] GET /identity/v3/role_assignments?user.id=9d90a7ed44fd4a1fa94b9d44939df425&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:08.088361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bdef6-66ce-4c16-b637-42cbfbdf8511 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:08.190640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:08.199508 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f60b89-0111-4b92-8d1e-a1df6d368f55 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:08.241694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f5e5b-2a43-4266-a27f-4fe55bbc0ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:08.242385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f5e5b-2a43-4266-a27f-4fe55bbc0ddf None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:08.242668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f5e5b-2a43-4266-a27f-4fe55bbc0ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:08.242941 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f5e5b-2a43-4266-a27f-4fe55bbc0ddf None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:08.243601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f5e5b-2a43-4266-a27f-4fe55bbc0ddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:08.243846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bdef6-66ce-4c16-b637-42cbfbdf8511 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:08.243962 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f5e5b-2a43-4266-a27f-4fe55bbc0ddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:08.245305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f5e5b-2a43-4266-a27f-4fe55bbc0ddf None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:08.260550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 355/685] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:08.262038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f5e5b-2a43-4266-a27f-4fe55bbc0ddf None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:08.292333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e940c-6992-4f14-bb3b-59108dfb20e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:08.292911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e940c-6992-4f14-bb3b-59108dfb20e4 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:08.293188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e940c-6992-4f14-bb3b-59108dfb20e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:08.293451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e940c-6992-4f14-bb3b-59108dfb20e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:08.297633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:08.307045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e940c-6992-4f14-bb3b-59108dfb20e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:08.308770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e940c-6992-4f14-bb3b-59108dfb20e4 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-169e940c-6992-4f14-bb3b-59108dfb20e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:08.326054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:08.327077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:08.328612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:08.328612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:08.328612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:08.328612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 331/686] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:08.328612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e940c-6992-4f14-bb3b-59108dfb20e4 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:08.402829 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f5e5b-2a43-4266-a27f-4fe55bbc0ddf None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:08.450968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0346b0c-63f7-46ea-9457-eef7b3e5dd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:08.451742 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0346b0c-63f7-46ea-9457-eef7b3e5dd55 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:08.452041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0346b0c-63f7-46ea-9457-eef7b3e5dd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:08.452345 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0346b0c-63f7-46ea-9457-eef7b3e5dd55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:08.453427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0346b0c-63f7-46ea-9457-eef7b3e5dd55 None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=aa222b1c89dc4d2cb1686b4838c7908b) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:08.453735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0346b0c-63f7-46ea-9457-eef7b3e5dd55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:08.455839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0346b0c-63f7-46ea-9457-eef7b3e5dd55 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:08.456884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e940c-6992-4f14-bb3b-59108dfb20e4 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:08.463785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 356/687] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:08 2022] GET /identity/v3/role_assignments?user.id=aa222b1c89dc4d2cb1686b4838c7908b&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:08.464671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0346b0c-63f7-46ea-9457-eef7b3e5dd55 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:08.490182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:08.527843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588e91c-e863-4c77-9e75-8781982b7ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:08.528540 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588e91c-e863-4c77-9e75-8781982b7ccb None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:08.528824 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588e91c-e863-4c77-9e75-8781982b7ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:08.529098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588e91c-e863-4c77-9e75-8781982b7ccb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:08.549126 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588e91c-e863-4c77-9e75-8781982b7ccb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:08.550675 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588e91c-e863-4c77-9e75-8781982b7ccb None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-9588e91c-e863-4c77-9e75-8781982b7ccb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:08.569988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:08.571018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:08.571942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:08.571942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:08.571942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:08.571942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:08.571942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:08.571942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:08.571942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:08.571942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:08.571942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:06:08.571942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:08.577277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 332/688] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:06:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:08.585173 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588e91c-e863-4c77-9e75-8781982b7ccb None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:08.597123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0346b0c-63f7-46ea-9457-eef7b3e5dd55 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:08.652874 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c30448-a1b8-4dad-8887-479681e258be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:08.653669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c30448-a1b8-4dad-8887-479681e258be None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:08.653851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c30448-a1b8-4dad-8887-479681e258be None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:08.654071 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c30448-a1b8-4dad-8887-479681e258be None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:08.654743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c30448-a1b8-4dad-8887-479681e258be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:08.655049 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c30448-a1b8-4dad-8887-479681e258be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:08.656774 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c30448-a1b8-4dad-8887-479681e258be None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:08.674503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 357/689] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:08.680366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c30448-a1b8-4dad-8887-479681e258be None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:08.710574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:08.764643 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588e91c-e863-4c77-9e75-8781982b7ccb None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:08.793759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:08.815038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f6c89-99f1-420e-b2cb-f928ccfb068f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:08.816109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f6c89-99f1-420e-b2cb-f928ccfb068f None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:08.816893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f6c89-99f1-420e-b2cb-f928ccfb068f None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:08.817323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f6c89-99f1-420e-b2cb-f928ccfb068f None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:08.818461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f6c89-99f1-420e-b2cb-f928ccfb068f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:08.819606 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f6c89-99f1-420e-b2cb-f928ccfb068f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:08.822458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f6c89-99f1-420e-b2cb-f928ccfb068f None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:08.823414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c30448-a1b8-4dad-8887-479681e258be None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:08.847251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 333/690] 10.222.0.85 () {56 vars in 1135 bytes} [Mon Aug 29 20:06:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:08.849333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f75d87-d2bb-4f8f-ac52-6de06cf2e0db None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:08.849623 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f75d87-d2bb-4f8f-ac52-6de06cf2e0db None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:08.849896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f75d87-d2bb-4f8f-ac52-6de06cf2e0db None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:08.851258 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 334/691] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:08.852989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f75d87-d2bb-4f8f-ac52-6de06cf2e0db None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:08.871076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89283b-50a1-4054-84c9-db7c56aa63e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:08.871544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89283b-50a1-4054-84c9-db7c56aa63e6 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:08.871823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89283b-50a1-4054-84c9-db7c56aa63e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:08.872303 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89283b-50a1-4054-84c9-db7c56aa63e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:08.880702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89283b-50a1-4054-84c9-db7c56aa63e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:08.882209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89283b-50a1-4054-84c9-db7c56aa63e6 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-ab89283b-50a1-4054-84c9-db7c56aa63e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:08.894949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:08.896065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:08.897350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:08.897350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:08.897350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:08.897350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:08.897350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:08.897350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:08.897350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:08.897350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:08.897350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:06:08.897350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:08.900947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 358/692] 10.222.0.85 () {56 vars in 1093 bytes} [Mon Aug 29 20:06:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:08.902215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611c21-2535-4328-b904-36fa79d09e98 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:08.902582 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611c21-2535-4328-b904-36fa79d09e98 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:08.902864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611c21-2535-4328-b904-36fa79d09e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:08.929640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-b1611c21-2535-4328-b904-36fa79d09e98 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:08.934443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b1611c21-2535-4328-b904-36fa79d09e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:08.986955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f75d87-d2bb-4f8f-ac52-6de06cf2e0db None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:08.995326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:09.023127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46689657-5b14-4234-b3e8-8d78af98409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:09.023784 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46689657-5b14-4234-b3e8-8d78af98409b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:09.024068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46689657-5b14-4234-b3e8-8d78af98409b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:09.024382 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46689657-5b14-4234-b3e8-8d78af98409b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:09.026092 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 359/693] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:06:09.027140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1611c21-2535-4328-b904-36fa79d09e98 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:09.103936 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46689657-5b14-4234-b3e8-8d78af98409b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:09.105524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46689657-5b14-4234-b3e8-8d78af98409b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:09.125661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:09.162429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1611c21-2535-4328-b904-36fa79d09e98 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:09.201133 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1844273-c4a0-4ec6-b956-44f2df7120b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:09.201783 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1844273-c4a0-4ec6-b956-44f2df7120b2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:09.202062 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1844273-c4a0-4ec6-b956-44f2df7120b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:09.202383 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1844273-c4a0-4ec6-b956-44f2df7120b2 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:09.203097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1844273-c4a0-4ec6-b956-44f2df7120b2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:09.203403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1844273-c4a0-4ec6-b956-44f2df7120b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:09.204952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1844273-c4a0-4ec6-b956-44f2df7120b2 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:09.231961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 335/694] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:06:09.238852 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46689657-5b14-4234-b3e8-8d78af98409b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:09.241551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 360/695] 10.222.0.85 () {56 vars in 1133 bytes} [Mon Aug 29 20:06:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:09.247391 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1844273-c4a0-4ec6-b956-44f2df7120b2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:09.252482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:09.376103 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46689657-5b14-4234-b3e8-8d78af98409b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:09.410901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1844273-c4a0-4ec6-b956-44f2df7120b2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:09.413192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc38d96-3747-40f3-84fb-2aff2dbdbf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:09.414798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc38d96-3747-40f3-84fb-2aff2dbdbf7f None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:09.414798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc38d96-3747-40f3-84fb-2aff2dbdbf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:09.414798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc38d96-3747-40f3-84fb-2aff2dbdbf7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:09.426194 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc38d96-3747-40f3-84fb-2aff2dbdbf7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:09.427660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc38d96-3747-40f3-84fb-2aff2dbdbf7f None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:09.440702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-0cc38d96-3747-40f3-84fb-2aff2dbdbf7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:09.443546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:09.444516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:09.445402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:09.445402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:09.445402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:09.445402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 336/696] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:09.446051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc38d96-3747-40f3-84fb-2aff2dbdbf7f None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:09.454634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66537392-5be1-4b48-aa46-2b67cf85798d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:09.455549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66537392-5be1-4b48-aa46-2b67cf85798d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:09.457367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66537392-5be1-4b48-aa46-2b67cf85798d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:09.457367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66537392-5be1-4b48-aa46-2b67cf85798d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:09.514115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:09.549925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66537392-5be1-4b48-aa46-2b67cf85798d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:09.552521 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66537392-5be1-4b48-aa46-2b67cf85798d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:09.563605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc38d96-3747-40f3-84fb-2aff2dbdbf7f None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:09.599719 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5f290-7888-4cbd-97cc-f888133c6420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:09.600410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5f290-7888-4cbd-97cc-f888133c6420 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:09.600699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5f290-7888-4cbd-97cc-f888133c6420 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:09.600974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5f290-7888-4cbd-97cc-f888133c6420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:09.613971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5f290-7888-4cbd-97cc-f888133c6420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:09.615607 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5f290-7888-4cbd-97cc-f888133c6420 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-50d5f290-7888-4cbd-97cc-f888133c6420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:09.631033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:09.632200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:09.633329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:09.633329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:09.633329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:09.633329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 337/697] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:09.633615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5f290-7888-4cbd-97cc-f888133c6420 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:09.692887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 361/698] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:06:09.694362 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66537392-5be1-4b48-aa46-2b67cf85798d None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:09.730320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:09.768596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5f290-7888-4cbd-97cc-f888133c6420 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:09.793315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:09.838679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37223c-f785-430d-99cc-3884242609fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:09.839766 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37223c-f785-430d-99cc-3884242609fc None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:09.840226 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37223c-f785-430d-99cc-3884242609fc None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:09.841076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37223c-f785-430d-99cc-3884242609fc None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:09.842059 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37223c-f785-430d-99cc-3884242609fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:09.842651 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37223c-f785-430d-99cc-3884242609fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:09.844896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37223c-f785-430d-99cc-3884242609fc None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:09.851309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66537392-5be1-4b48-aa46-2b67cf85798d None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:09.866290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 338/699] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:09.866290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37223c-f785-430d-99cc-3884242609fc None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:09.894430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63755d98-eb51-4a7c-a77c-8a2df109f512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:09.895042 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63755d98-eb51-4a7c-a77c-8a2df109f512 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:09.895322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63755d98-eb51-4a7c-a77c-8a2df109f512 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:09.895643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63755d98-eb51-4a7c-a77c-8a2df109f512 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:09.896334 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63755d98-eb51-4a7c-a77c-8a2df109f512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:09.896640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63755d98-eb51-4a7c-a77c-8a2df109f512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:09.898067 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63755d98-eb51-4a7c-a77c-8a2df109f512 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:09.911602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 362/700] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:09.912596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63755d98-eb51-4a7c-a77c-8a2df109f512 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:10.007629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:10.010055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37223c-f785-430d-99cc-3884242609fc None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:10.024464 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63755d98-eb51-4a7c-a77c-8a2df109f512 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:10.059724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4267cbc-958e-4997-8e45-fd291a64a96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:10.060698 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4267cbc-958e-4997-8e45-fd291a64a96b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:10.061191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4267cbc-958e-4997-8e45-fd291a64a96b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:10.061647 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4267cbc-958e-4997-8e45-fd291a64a96b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:10.063004 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e95ef4-9396-40d1-8c2d-eeef4ae860b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:10.063786 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e95ef4-9396-40d1-8c2d-eeef4ae860b7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:10.064144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e95ef4-9396-40d1-8c2d-eeef4ae860b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:10.064516 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e95ef4-9396-40d1-8c2d-eeef4ae860b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:10.072208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e95ef4-9396-40d1-8c2d-eeef4ae860b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:10.073828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e95ef4-9396-40d1-8c2d-eeef4ae860b7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:10.075570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4267cbc-958e-4997-8e45-fd291a64a96b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:10.078016 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4267cbc-958e-4997-8e45-fd291a64a96b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-41e95ef4-9396-40d1-8c2d-eeef4ae860b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:10.086880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:10.087841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:10.088850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:10.088850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:10.088850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:10.088850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:10.088850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:10.088850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:10.088850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:10.088850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:10.093157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 363/701] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:10.094753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e95ef4-9396-40d1-8c2d-eeef4ae860b7 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-f4267cbc-958e-4997-8e45-fd291a64a96b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:10.098397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:10.099319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:10.100268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:10.100268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:10.100268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:10.104246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 339/702] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:10.105296 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4267cbc-958e-4997-8e45-fd291a64a96b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:10.142947 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:10.227978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4267cbc-958e-4997-8e45-fd291a64a96b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:10.233200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e95ef4-9396-40d1-8c2d-eeef4ae860b7 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:10.278586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed5ce2-ed50-4ed7-975c-837485baf2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:10.278933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4c185-d045-4ab0-bb05-ff031c31dea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:10.279155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed5ce2-ed50-4ed7-975c-837485baf2f2 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:10.279439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed5ce2-ed50-4ed7-975c-837485baf2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:10.279712 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed5ce2-ed50-4ed7-975c-837485baf2f2 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:10.280384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed5ce2-ed50-4ed7-975c-837485baf2f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:10.280661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4c185-d045-4ab0-bb05-ff031c31dea0 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:10.281418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4c185-d045-4ab0-bb05-ff031c31dea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:10.281900 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4c185-d045-4ab0-bb05-ff031c31dea0 None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:10.282007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed5ce2-ed50-4ed7-975c-837485baf2f2 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:10.282911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4c185-d045-4ab0-bb05-ff031c31dea0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:10.283436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4c185-d045-4ab0-bb05-ff031c31dea0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:10.285093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4c185-d045-4ab0-bb05-ff031c31dea0 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:10.301050 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 340/703] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:10.302270 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4c185-d045-4ab0-bb05-ff031c31dea0 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:10.357021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 364/704] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:06:10 2022] POST /identity/v3/users => generated 254 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:06:10.358232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed5ce2-ed50-4ed7-975c-837485baf2f2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:10.372558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:10.405088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:10.427288 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4c185-d045-4ab0-bb05-ff031c31dea0 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:10.477537 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ab6a7-dbd2-43fa-87a9-ab9070d0f33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:10.478604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ab6a7-dbd2-43fa-87a9-ab9070d0f33e None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:10.479449 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ab6a7-dbd2-43fa-87a9-ab9070d0f33e None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:10.480143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ab6a7-dbd2-43fa-87a9-ab9070d0f33e None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:10.481001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ab6a7-dbd2-43fa-87a9-ab9070d0f33e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:10.481709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ab6a7-dbd2-43fa-87a9-ab9070d0f33e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:10.484228 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ab6a7-dbd2-43fa-87a9-ab9070d0f33e None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:10.490300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed5ce2-ed50-4ed7-975c-837485baf2f2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:10.500490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 341/705] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:10.502466 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365346c-d2d1-47c5-a62b-352f88184a7d None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:10.502905 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365346c-d2d1-47c5-a62b-352f88184a7d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:10.503715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365346c-d2d1-47c5-a62b-352f88184a7d None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:10.505469 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 342/706] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:10.508325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365346c-d2d1-47c5-a62b-352f88184a7d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:10.545950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb14498-e9d3-4ebb-91cd-e672f46ab2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:10.546560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb14498-e9d3-4ebb-91cd-e672f46ab2cf None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:10.546897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb14498-e9d3-4ebb-91cd-e672f46ab2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:10.547257 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb14498-e9d3-4ebb-91cd-e672f46ab2cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:10.548060 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb14498-e9d3-4ebb-91cd-e672f46ab2cf None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=d52f84245a2142e69c309aa709a7d921) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:10.548406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb14498-e9d3-4ebb-91cd-e672f46ab2cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:10.549931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb14498-e9d3-4ebb-91cd-e672f46ab2cf None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:10.559141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 365/707] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:10 2022] GET /identity/v3/role_assignments?user.id=d52f84245a2142e69c309aa709a7d921&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:10.561333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86c5bf-3d7a-456f-a1a8-5bc06787d303 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:10.561626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86c5bf-3d7a-456f-a1a8-5bc06787d303 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:10.561910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86c5bf-3d7a-456f-a1a8-5bc06787d303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:10.588262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-2e86c5bf-3d7a-456f-a1a8-5bc06787d303 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:10.589548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2e86c5bf-3d7a-456f-a1a8-5bc06787d303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:10.663237 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:10.671741 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 366/708] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:06:10.673270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d25ce6-5277-4543-ab4d-7366e2f5d705 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:10.673560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d25ce6-5277-4543-ab4d-7366e2f5d705 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:10.679260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d25ce6-5277-4543-ab4d-7366e2f5d705 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:10.679260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 367/709] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:10.681393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7b27a-0e63-490f-b26d-faa8384a73b0 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:10.681678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7b27a-0e63-490f-b26d-faa8384a73b0 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:10.681948 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7b27a-0e63-490f-b26d-faa8384a73b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:10.685590 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365346c-d2d1-47c5-a62b-352f88184a7d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:10.702547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-94b7b27a-0e63-490f-b26d-faa8384a73b0 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:10.704450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-94b7b27a-0e63-490f-b26d-faa8384a73b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:10.754250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539b5c1-fac0-4074-a456-2395ddb27807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:10.754891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539b5c1-fac0-4074-a456-2395ddb27807 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:10.755170 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539b5c1-fac0-4074-a456-2395ddb27807 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:10.755444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539b5c1-fac0-4074-a456-2395ddb27807 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:10.763369 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539b5c1-fac0-4074-a456-2395ddb27807 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:10.765278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539b5c1-fac0-4074-a456-2395ddb27807 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-b539b5c1-fac0-4074-a456-2395ddb27807 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:10.778617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:10.779589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:10.780599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:10.780599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:10.780599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:10.780599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:10.780599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:10.780599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:10.780599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:10.780599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:10.780599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 343/710] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:10.787687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539b5c1-fac0-4074-a456-2395ddb27807 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:10.790538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:10.794858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 368/711] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:06:10.907220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539b5c1-fac0-4074-a456-2395ddb27807 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:10.944474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace440d4-854e-4b56-b86c-0647880d8655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:10.945084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace440d4-854e-4b56-b86c-0647880d8655 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:10.945366 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace440d4-854e-4b56-b86c-0647880d8655 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:10.945639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace440d4-854e-4b56-b86c-0647880d8655 None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:10.946338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace440d4-854e-4b56-b86c-0647880d8655 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:10.946646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace440d4-854e-4b56-b86c-0647880d8655 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:10.948075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace440d4-854e-4b56-b86c-0647880d8655 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:10.962055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 344/712] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:10.967881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7b27a-0e63-490f-b26d-faa8384a73b0 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:10.987523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace440d4-854e-4b56-b86c-0647880d8655 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:11.058070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:11.102746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace440d4-854e-4b56-b86c-0647880d8655 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:11.149252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b915e-cab3-4e76-8557-84407df7cd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:11.149883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b915e-cab3-4e76-8557-84407df7cd69 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:11.150422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b915e-cab3-4e76-8557-84407df7cd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:11.150692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b915e-cab3-4e76-8557-84407df7cd69 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:11.151266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b915e-cab3-4e76-8557-84407df7cd69 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:11.152664 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b915e-cab3-4e76-8557-84407df7cd69 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-890b915e-cab3-4e76-8557-84407df7cd69 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:11.164609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:11.165734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:06:11.166662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:11.170525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 345/713] 10.222.0.85 () {56 vars in 1129 bytes} [Mon Aug 29 20:06:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:11.171609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b915e-cab3-4e76-8557-84407df7cd69 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:11.199491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7b27a-0e63-490f-b26d-faa8384a73b0 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:11.229942 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:11.240354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f343a-0290-420b-82c7-dab6d38d70bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:11.241326 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f343a-0290-420b-82c7-dab6d38d70bf None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:11.241776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f343a-0290-420b-82c7-dab6d38d70bf None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:11.242216 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f343a-0290-420b-82c7-dab6d38d70bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:11.246491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f343a-0290-420b-82c7-dab6d38d70bf None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=805df3047fee45c7a702d72a5c082eb5) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:11.246491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f343a-0290-420b-82c7-dab6d38d70bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:11.246491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f343a-0290-420b-82c7-dab6d38d70bf None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:11.252587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 369/714] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:10 2022] GET /identity/v3/role_assignments?user.id=805df3047fee45c7a702d72a5c082eb5&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:11.253766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f343a-0290-420b-82c7-dab6d38d70bf None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:11.278302 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b915e-cab3-4e76-8557-84407df7cd69 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:11.308338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:11.318175 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca69a48-9edf-4cae-9bb0-9d88976af16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:11.318858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca69a48-9edf-4cae-9bb0-9d88976af16a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:11.319191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca69a48-9edf-4cae-9bb0-9d88976af16a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:11.326698 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca69a48-9edf-4cae-9bb0-9d88976af16a None admin] PATH_INFO: `/v3/services/network` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:11.326698 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca69a48-9edf-4cae-9bb0-9d88976af16a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:11.326698 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca69a48-9edf-4cae-9bb0-9d88976af16a None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-4ca69a48-9edf-4cae-9bb0-9d88976af16a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:11.331255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:11.332278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:11.333123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 346/715] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:11 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:11.343808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca69a48-9edf-4cae-9bb0-9d88976af16a None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:11.376862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f343a-0290-420b-82c7-dab6d38d70bf None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:11.414571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd72c2-69e4-4801-9fad-f0ab02775513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:11.415664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd72c2-69e4-4801-9fad-f0ab02775513 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:11.416160 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd72c2-69e4-4801-9fad-f0ab02775513 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:11.416678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd72c2-69e4-4801-9fad-f0ab02775513 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:11.417541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd72c2-69e4-4801-9fad-f0ab02775513 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:11.418134 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd72c2-69e4-4801-9fad-f0ab02775513 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:11.419864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd72c2-69e4-4801-9fad-f0ab02775513 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:11.425849 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 370/716] 10.222.0.85 () {56 vars in 1124 bytes} [Mon Aug 29 20:06:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:11.434574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd72c2-69e4-4801-9fad-f0ab02775513 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:11.496061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca69a48-9edf-4cae-9bb0-9d88976af16a None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:11.500881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:11.533928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53368c82-42ad-4f47-82ec-6373720e15ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:11.534532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53368c82-42ad-4f47-82ec-6373720e15ca None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:11.534805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53368c82-42ad-4f47-82ec-6373720e15ca None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:11.535070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53368c82-42ad-4f47-82ec-6373720e15ca None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:11.535689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53368c82-42ad-4f47-82ec-6373720e15ca None admin] RBAC: Adding query filter params (name=network) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:11.535988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53368c82-42ad-4f47-82ec-6373720e15ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:11.537451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53368c82-42ad-4f47-82ec-6373720e15ca None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:11.542535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 347/717] 10.222.0.85 () {56 vars in 1112 bytes} [Mon Aug 29 20:06:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:11.548382 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53368c82-42ad-4f47-82ec-6373720e15ca None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:11.559864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd72c2-69e4-4801-9fad-f0ab02775513 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:11.622527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:11.665475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53368c82-42ad-4f47-82ec-6373720e15ca None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:11.696987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f718f8-57ec-419c-88f3-804a0f708ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:11.697536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f718f8-57ec-419c-88f3-804a0f708ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:11.697821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f718f8-57ec-419c-88f3-804a0f708ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:11.698102 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f718f8-57ec-419c-88f3-804a0f708ab3 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:11.698746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f718f8-57ec-419c-88f3-804a0f708ab3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:11.699051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f718f8-57ec-419c-88f3-804a0f708ab3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:11.700450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f718f8-57ec-419c-88f3-804a0f708ab3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:11.702152 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89de00-c95e-4448-a06e-52dfd5a5a678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:11.703643 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89de00-c95e-4448-a06e-52dfd5a5a678 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:11.704066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89de00-c95e-4448-a06e-52dfd5a5a678 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:11.704518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89de00-c95e-4448-a06e-52dfd5a5a678 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:11.705442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89de00-c95e-4448-a06e-52dfd5a5a678 None admin] RBAC: Adding query filter params (type=network) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:11.705950 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89de00-c95e-4448-a06e-52dfd5a5a678 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:11.707465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89de00-c95e-4448-a06e-52dfd5a5a678 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:11.714838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 348/718] 10.222.0.85 () {56 vars in 1112 bytes} [Mon Aug 29 20:06:11 2022] GET /identity/v3/services?type=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:11.726512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 371/719] 10.222.0.85 () {56 vars in 1124 bytes} [Mon Aug 29 20:06:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:12.656353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56949174-3585-41a7-b5af-37ad8143ade0 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:12.656875 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56949174-3585-41a7-b5af-37ad8143ade0 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:12.656875 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56949174-3585-41a7-b5af-37ad8143ade0 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:12.658491 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 349/720] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:12.664095 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d5fec-16f9-4d7e-81be-a03311685f91 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:12.664415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d5fec-16f9-4d7e-81be-a03311685f91 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:12.664677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d5fec-16f9-4d7e-81be-a03311685f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:12.689128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-ca8d5fec-16f9-4d7e-81be-a03311685f91 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:12.690318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ca8d5fec-16f9-4d7e-81be-a03311685f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:12.764458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 372/721] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:06:13.157703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56949174-3585-41a7-b5af-37ad8143ade0 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:13.309189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56949174-3585-41a7-b5af-37ad8143ade0 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:13.330816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:13.355616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883a01a-cf5f-4737-84e2-84f3dfc82481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:13.356332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883a01a-cf5f-4737-84e2-84f3dfc82481 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:13.356614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883a01a-cf5f-4737-84e2-84f3dfc82481 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:13.356884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883a01a-cf5f-4737-84e2-84f3dfc82481 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:13.374088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883a01a-cf5f-4737-84e2-84f3dfc82481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-3883a01a-cf5f-4737-84e2-84f3dfc82481 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:13.379435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:13.380545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:13.381383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:13.381383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:13.381383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:13.381383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:13.381383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:13.381383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:13.381383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:13.381383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:13.381383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:13.381383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:13.381383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:13.381383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:13.381383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 350/722] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:06:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:13.386641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d5fec-16f9-4d7e-81be-a03311685f91 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:13.415546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3ad06-cbff-49eb-86d9-1246185d99a3 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:13.415841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3ad06-cbff-49eb-86d9-1246185d99a3 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:13.416006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3ad06-cbff-49eb-86d9-1246185d99a3 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:13.417629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 351/723] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:13.423304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990487b-15d4-4794-bcaa-69bd14d713ef None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:13.423580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990487b-15d4-4794-bcaa-69bd14d713ef None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:13.423858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990487b-15d4-4794-bcaa-69bd14d713ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:13.477667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b990487b-15d4-4794-bcaa-69bd14d713ef None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:13.478711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b990487b-15d4-4794-bcaa-69bd14d713ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:13.488639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:13.513533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d5fec-16f9-4d7e-81be-a03311685f91 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:13.553760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d9797-299b-44d0-aee6-f3e0dd26aad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:13.554388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d9797-299b-44d0-aee6-f3e0dd26aad9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:13.554667 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d9797-299b-44d0-aee6-f3e0dd26aad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:13.554997 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d9797-299b-44d0-aee6-f3e0dd26aad9 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:13.555666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d9797-299b-44d0-aee6-f3e0dd26aad9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:13.555965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d9797-299b-44d0-aee6-f3e0dd26aad9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:13.557762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d9797-299b-44d0-aee6-f3e0dd26aad9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:13.568023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 373/724] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:06:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:13.573035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d9797-299b-44d0-aee6-f3e0dd26aad9 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:13.625246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 352/725] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:06:13.663944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079c3a2-b183-43ea-b6f9-fd083f8c59b4 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:13.665015 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079c3a2-b183-43ea-b6f9-fd083f8c59b4 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:13.665476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079c3a2-b183-43ea-b6f9-fd083f8c59b4 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:13.667317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 353/726] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:13.672691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ee5d0-97d3-4fef-93fd-aa2d3d91e62a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:13.673269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ee5d0-97d3-4fef-93fd-aa2d3d91e62a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:13.673749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ee5d0-97d3-4fef-93fd-aa2d3d91e62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:13.692732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-bc6ee5d0-97d3-4fef-93fd-aa2d3d91e62a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:13.694283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bc6ee5d0-97d3-4fef-93fd-aa2d3d91e62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:13.705084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d9797-299b-44d0-aee6-f3e0dd26aad9 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:13.731801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:13.746686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9428c-8dc9-41a1-83ac-7b982b0164e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:13.747831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9428c-8dc9-41a1-83ac-7b982b0164e6 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:13.748362 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9428c-8dc9-41a1-83ac-7b982b0164e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:13.748847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9428c-8dc9-41a1-83ac-7b982b0164e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:13.763107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9428c-8dc9-41a1-83ac-7b982b0164e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:13.765255 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9428c-8dc9-41a1-83ac-7b982b0164e6 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-fbc9428c-8dc9-41a1-83ac-7b982b0164e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:13.790735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:13.791771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:13.792746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:13.792746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:13.792746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:13.796248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 374/727] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:13.801995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9428c-8dc9-41a1-83ac-7b982b0164e6 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:13.806775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 354/728] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:06:13.941377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9428c-8dc9-41a1-83ac-7b982b0164e6 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:14.014737 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601d4c0-5b11-400e-99d0-d4cab2836a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:14.015872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601d4c0-5b11-400e-99d0-d4cab2836a48 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.016152 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601d4c0-5b11-400e-99d0-d4cab2836a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.016451 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601d4c0-5b11-400e-99d0-d4cab2836a48 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.017093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601d4c0-5b11-400e-99d0-d4cab2836a48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:14.017390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601d4c0-5b11-400e-99d0-d4cab2836a48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:14.018760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601d4c0-5b11-400e-99d0-d4cab2836a48 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:14.031732 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 375/729] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:14.036864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ee5d0-97d3-4fef-93fd-aa2d3d91e62a None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:14.063044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601d4c0-5b11-400e-99d0-d4cab2836a48 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:14.106091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:14.112538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:14.178930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ee5d0-97d3-4fef-93fd-aa2d3d91e62a None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:14.186703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601d4c0-5b11-400e-99d0-d4cab2836a48 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:14.221267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486648da-d7b3-410e-8992-669e875b080d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:14.221900 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486648da-d7b3-410e-8992-669e875b080d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.222180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486648da-d7b3-410e-8992-669e875b080d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.222447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486648da-d7b3-410e-8992-669e875b080d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.222737 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545ff0c-0e14-493c-afed-9365751f3852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:14.223367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545ff0c-0e14-493c-afed-9365751f3852 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.223649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545ff0c-0e14-493c-afed-9365751f3852 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.223925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545ff0c-0e14-493c-afed-9365751f3852 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.230603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486648da-d7b3-410e-8992-669e875b080d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:14.232058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486648da-d7b3-410e-8992-669e875b080d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:14.235379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545ff0c-0e14-493c-afed-9365751f3852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-e545ff0c-0e14-493c-afed-9365751f3852 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:14.240343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:14.241424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:14.242284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:14.242284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:14.242284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:14.242284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:14.242284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:14.242284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:14.242284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:14.242284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:14.242284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:14.242284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:14.242284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:14.242284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:14.242284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 376/730] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:14.243394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7954235-7032-446d-9ffc-68e12e8c70ba None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.243394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7954235-7032-446d-9ffc-68e12e8c70ba None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.243561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7954235-7032-446d-9ffc-68e12e8c70ba None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.244977 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 377/731] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-486648da-d7b3-410e-8992-669e875b080d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:14.245080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:14.246423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:14.249017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:14.249017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:14.249017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:14.249017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:14.249017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:14.249017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:14.249017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:14.249017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:14.249017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:06:14.249017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:14.249017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 355/732] 10.222.0.85 () {56 vars in 1117 bytes} [Mon Aug 29 20:06:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:14.249017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5d82-8968-48ad-bb3b-169b4081fb17 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.249017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5d82-8968-48ad-bb3b-169b4081fb17 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.251616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5d82-8968-48ad-bb3b-169b4081fb17 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.251859 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7954235-7032-446d-9ffc-68e12e8c70ba None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:14.252007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 356/733] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:14.252007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be5d82-8968-48ad-bb3b-169b4081fb17 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:14.339187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:14.349419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:14.354058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be5d82-8968-48ad-bb3b-169b4081fb17 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:14.357283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7954235-7032-446d-9ffc-68e12e8c70ba None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:14.387712 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bdfefc-4674-4b73-b4be-8d58446bc0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:14.388338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdfefc-4674-4b73-b4be-8d58446bc0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.388627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdfefc-4674-4b73-b4be-8d58446bc0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.388893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdfefc-4674-4b73-b4be-8d58446bc0e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.389554 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bdfefc-4674-4b73-b4be-8d58446bc0e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:14.389855 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bdfefc-4674-4b73-b4be-8d58446bc0e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:14.391822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bdfefc-4674-4b73-b4be-8d58446bc0e2 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:14.392452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f2500-59ee-4e16-a90c-295c29bcdf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:14.393079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f2500-59ee-4e16-a90c-295c29bcdf7e None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.393352 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f2500-59ee-4e16-a90c-295c29bcdf7e None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.393624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f2500-59ee-4e16-a90c-295c29bcdf7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.399946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 357/734] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:14.401392 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322d10f-d32c-40b0-ac12-e014be9fc8f6 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.401676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322d10f-d32c-40b0-ac12-e014be9fc8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.401947 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322d10f-d32c-40b0-ac12-e014be9fc8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.404273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f2500-59ee-4e16-a90c-295c29bcdf7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-f05f2500-59ee-4e16-a90c-295c29bcdf7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:14.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:14.410467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:14.411319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:14.411319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:14.411319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:14.411319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:14.411319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:14.411319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:14.411319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:14.411319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:14.411319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:14.411319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:14.411319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:14.411319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:14.411319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 378/735] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:14.412097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f2500-59ee-4e16-a90c-295c29bcdf7e None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:14.422269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-c322d10f-d32c-40b0-ac12-e014be9fc8f6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:14.423480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c322d10f-d32c-40b0-ac12-e014be9fc8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:14.490898 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 358/736] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:06:14.491956 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3cd83-43b0-4b7f-98de-c94360946251 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.492276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3cd83-43b0-4b7f-98de-c94360946251 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.492555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3cd83-43b0-4b7f-98de-c94360946251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.511490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3cf3cd83-43b0-4b7f-98de-c94360946251 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:14.513056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3cf3cd83-43b0-4b7f-98de-c94360946251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:14.522046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f2500-59ee-4e16-a90c-295c29bcdf7e None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:14.556443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fba79-068c-4f61-b5e8-161899304f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:14.557055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fba79-068c-4f61-b5e8-161899304f77 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.557389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fba79-068c-4f61-b5e8-161899304f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.557607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fba79-068c-4f61-b5e8-161899304f77 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.558342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fba79-068c-4f61-b5e8-161899304f77 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:14.558647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fba79-068c-4f61-b5e8-161899304f77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:14.560193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fba79-068c-4f61-b5e8-161899304f77 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:14.574099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:14.578111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 379/737] 10.222.0.85 () {56 vars in 1149 bytes} [Mon Aug 29 20:06:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:14.578990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fba79-068c-4f61-b5e8-161899304f77 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:14.592700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 359/738] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:06:14.594193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3cd83-43b0-4b7f-98de-c94360946251 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:14.622209 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:14.701883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fba79-068c-4f61-b5e8-161899304f77 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:14.708485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3cd83-43b0-4b7f-98de-c94360946251 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:14.739813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd16c0-5cd6-41da-aea0-5c103d4c8472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:14.741094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd16c0-5cd6-41da-aea0-5c103d4c8472 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.741819 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd16c0-5cd6-41da-aea0-5c103d4c8472 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.742243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd16c0-5cd6-41da-aea0-5c103d4c8472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.745971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd0fac-c994-4f16-bab0-b5aaea3fe818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:14.746699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd0fac-c994-4f16-bab0-b5aaea3fe818 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.747085 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd0fac-c994-4f16-bab0-b5aaea3fe818 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.747468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd0fac-c994-4f16-bab0-b5aaea3fe818 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.748338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd0fac-c994-4f16-bab0-b5aaea3fe818 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:14.748794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd0fac-c994-4f16-bab0-b5aaea3fe818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:14.750861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd0fac-c994-4f16-bab0-b5aaea3fe818 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:14.754986 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd16c0-5cd6-41da-aea0-5c103d4c8472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:14.757099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd16c0-5cd6-41da-aea0-5c103d4c8472 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:14.760141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 360/739] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:14.765714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd0fac-c994-4f16-bab0-b5aaea3fe818 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-16fd16c0-5cd6-41da-aea0-5c103d4c8472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:14.775857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:14.777196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:14.778063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:14.778063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:14.778063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:14.778063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 380/740] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:14.779021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd16c0-5cd6-41da-aea0-5c103d4c8472 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:14.833447 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:14.859819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:14.879010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd0fac-c994-4f16-bab0-b5aaea3fe818 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:14.882659 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd16c0-5cd6-41da-aea0-5c103d4c8472 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:14.912257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d351d41-9967-49c2-bbb1-b722aa315f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:14.912858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d351d41-9967-49c2-bbb1-b722aa315f23 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.913140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d351d41-9967-49c2-bbb1-b722aa315f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.913415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d351d41-9967-49c2-bbb1-b722aa315f23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.915896 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd9529-7fbc-4767-abcd-895ce55ceb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:14.916524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd9529-7fbc-4767-abcd-895ce55ceb12 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:14.916802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd9529-7fbc-4767-abcd-895ce55ceb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:14.917083 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd9529-7fbc-4767-abcd-895ce55ceb12 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:14.917728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd9529-7fbc-4767-abcd-895ce55ceb12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:14.919103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd9529-7fbc-4767-abcd-895ce55ceb12 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:14.981676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d351d41-9967-49c2-bbb1-b722aa315f23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:14.983296 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d351d41-9967-49c2-bbb1-b722aa315f23 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:15.060259 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 381/741] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:06:14 2022] POST /identity/v3/services => generated 240 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:06:15.061024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd9529-7fbc-4767-abcd-895ce55ceb12 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:15.065626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:15.091288 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 361/742] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:06:15.092601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d351d41-9967-49c2-bbb1-b722aa315f23 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:15.206574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d351d41-9967-49c2-bbb1-b722aa315f23 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:15.210102 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd9529-7fbc-4767-abcd-895ce55ceb12 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:15.222483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:15.245314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f82d4-8a72-432b-9b07-35951492fff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:15.245901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f82d4-8a72-432b-9b07-35951492fff2 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:15.246180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f82d4-8a72-432b-9b07-35951492fff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:15.246516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f82d4-8a72-432b-9b07-35951492fff2 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:15.247154 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f82d4-8a72-432b-9b07-35951492fff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:15.247453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f82d4-8a72-432b-9b07-35951492fff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:15.248844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f82d4-8a72-432b-9b07-35951492fff2 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:15.250756 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abce6e-d678-44d0-8132-398773ae6ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:15.251403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abce6e-d678-44d0-8132-398773ae6ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:15.251680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abce6e-d678-44d0-8132-398773ae6ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:15.251951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abce6e-d678-44d0-8132-398773ae6ac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:15.262525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79abce6e-d678-44d0-8132-398773ae6ac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:15.263066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 362/743] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:15.264220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f82d4-8a72-432b-9b07-35951492fff2 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:15.264378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79abce6e-d678-44d0-8132-398773ae6ac5 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-79abce6e-d678-44d0-8132-398773ae6ac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:15.281621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:15.282821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:15.283856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:15.283856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:15.283856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:15.283856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 382/744] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:15.284079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abce6e-d678-44d0-8132-398773ae6ac5 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:15.318953 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:15.377195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f82d4-8a72-432b-9b07-35951492fff2 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:15.393177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abce6e-d678-44d0-8132-398773ae6ac5 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:15.416660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba77347-b5da-46c6-bd4b-57d7722e4165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:15.417374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba77347-b5da-46c6-bd4b-57d7722e4165 None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:15.417674 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba77347-b5da-46c6-bd4b-57d7722e4165 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:15.418125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba77347-b5da-46c6-bd4b-57d7722e4165 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:15.418657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba77347-b5da-46c6-bd4b-57d7722e4165 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:15.420144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba77347-b5da-46c6-bd4b-57d7722e4165 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:15.429249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417ec96-1558-42e4-a645-60fb87a356df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:15.429885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417ec96-1558-42e4-a645-60fb87a356df None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:15.430163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417ec96-1558-42e4-a645-60fb87a356df None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:15.430434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417ec96-1558-42e4-a645-60fb87a356df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:15.441130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417ec96-1558-42e4-a645-60fb87a356df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:15.442576 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417ec96-1558-42e4-a645-60fb87a356df None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-7417ec96-1558-42e4-a645-60fb87a356df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:15.457660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:15.458629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:15.459523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:15.459523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:15.459523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:15.459523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 383/745] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:15.460294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417ec96-1558-42e4-a645-60fb87a356df None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:15.475707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:15.506401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 363/746] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:06:15 2022] POST /identity/v3/services => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:06:15.517299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba77347-b5da-46c6-bd4b-57d7722e4165 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:15.580035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417ec96-1558-42e4-a645-60fb87a356df None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:15.641440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:15.646705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba77347-b5da-46c6-bd4b-57d7722e4165 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:15.652136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d3f6c-086f-44cc-abb3-6b456220461d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:15.653104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d3f6c-086f-44cc-abb3-6b456220461d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:15.653576 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d3f6c-086f-44cc-abb3-6b456220461d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:15.654001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d3f6c-086f-44cc-abb3-6b456220461d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:15.664491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d3f6c-086f-44cc-abb3-6b456220461d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:15.665950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d3f6c-086f-44cc-abb3-6b456220461d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-ca4d3f6c-086f-44cc-abb3-6b456220461d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:15.679599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:15.680602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:15.681517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:15.681517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:15.681517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:15.681517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:15.681517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:15.681517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:15.681517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:15.681517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:15.681517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:06:15.681517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:15.684041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 384/747] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:06:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:15.685218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d3f6c-086f-44cc-abb3-6b456220461d None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:15.691488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225f16f-d82c-498c-a62e-250576b1a25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:15.692329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225f16f-d82c-498c-a62e-250576b1a25a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:15.692987 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225f16f-d82c-498c-a62e-250576b1a25a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:15.693447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225f16f-d82c-498c-a62e-250576b1a25a None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:15.694283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225f16f-d82c-498c-a62e-250576b1a25a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:15.694595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225f16f-d82c-498c-a62e-250576b1a25a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:15.695980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225f16f-d82c-498c-a62e-250576b1a25a None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:15.709687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 364/748] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:15.710410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225f16f-d82c-498c-a62e-250576b1a25a None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:15.763449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:15.802942 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d3f6c-086f-44cc-abb3-6b456220461d None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:15.845417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225f16f-d82c-498c-a62e-250576b1a25a None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:15.849323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f0fd2-dadd-4e13-ba47-918af11fec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:15.850077 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f0fd2-dadd-4e13-ba47-918af11fec01 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:15.850460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f0fd2-dadd-4e13-ba47-918af11fec01 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:15.851298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f0fd2-dadd-4e13-ba47-918af11fec01 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:15.852072 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f0fd2-dadd-4e13-ba47-918af11fec01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:15.852510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f0fd2-dadd-4e13-ba47-918af11fec01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:15.853982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f0fd2-dadd-4e13-ba47-918af11fec01 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:15.871342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 385/749] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:15.872890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f0fd2-dadd-4e13-ba47-918af11fec01 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:15.890524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965eb65-b28f-44dd-a897-8f551404ed98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:15.891281 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965eb65-b28f-44dd-a897-8f551404ed98 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:15.892144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965eb65-b28f-44dd-a897-8f551404ed98 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:15.892608 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965eb65-b28f-44dd-a897-8f551404ed98 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:15.893826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965eb65-b28f-44dd-a897-8f551404ed98 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:15.894237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965eb65-b28f-44dd-a897-8f551404ed98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:15.896461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965eb65-b28f-44dd-a897-8f551404ed98 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:15.927485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 365/750] 10.222.0.85 () {56 vars in 1135 bytes} [Mon Aug 29 20:06:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:15.928623 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965eb65-b28f-44dd-a897-8f551404ed98 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:15.954372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:16.000123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f0fd2-dadd-4e13-ba47-918af11fec01 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:16.036369 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:16.045502 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14398e51-2228-43b4-8eb1-1acad07a6f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:16.046470 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14398e51-2228-43b4-8eb1-1acad07a6f3a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:16.046811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14398e51-2228-43b4-8eb1-1acad07a6f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:16.051214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14398e51-2228-43b4-8eb1-1acad07a6f3a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:16.052534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965eb65-b28f-44dd-a897-8f551404ed98 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:16.055727 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14398e51-2228-43b4-8eb1-1acad07a6f3a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:16.057142 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14398e51-2228-43b4-8eb1-1acad07a6f3a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-14398e51-2228-43b4-8eb1-1acad07a6f3a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:16.070936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:16.072293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:16.073603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:16.073603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:16.073603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:16.073603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:16.073603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:16.073603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:16.073603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:16.073603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:16.073603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:06:16.073603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:16.075385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 386/751] 10.222.0.85 () {56 vars in 1093 bytes} [Mon Aug 29 20:06:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:16.077184 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14398e51-2228-43b4-8eb1-1acad07a6f3a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:16.095653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9ece4-1166-4bf2-aa2e-62fe6e3349c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:16.096639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9ece4-1166-4bf2-aa2e-62fe6e3349c3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:16.097178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9ece4-1166-4bf2-aa2e-62fe6e3349c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:16.097633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9ece4-1166-4bf2-aa2e-62fe6e3349c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:16.106880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9ece4-1166-4bf2-aa2e-62fe6e3349c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:16.109453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9ece4-1166-4bf2-aa2e-62fe6e3349c3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-99a9ece4-1166-4bf2-aa2e-62fe6e3349c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:16.125782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:16.126796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:16.127696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:16.127696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:16.127696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:16.127696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:16.127696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:16.127696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:16.127696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:16.127696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:16.130692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 366/752] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:16.132157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9ece4-1166-4bf2-aa2e-62fe6e3349c3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:16.201894 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14398e51-2228-43b4-8eb1-1acad07a6f3a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:16.240530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6398c4d-5b8f-40a5-af0c-2fa95f3ff699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:16.241887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6398c4d-5b8f-40a5-af0c-2fa95f3ff699 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:16.242361 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6398c4d-5b8f-40a5-af0c-2fa95f3ff699 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:16.242843 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6398c4d-5b8f-40a5-af0c-2fa95f3ff699 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:16.243542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:16.273922 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9ece4-1166-4bf2-aa2e-62fe6e3349c3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:16.317978 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:16.319664 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09e18c-a94d-49a1-955b-c72a1dfad361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:16.320727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09e18c-a94d-49a1-955b-c72a1dfad361 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:16.321224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09e18c-a94d-49a1-955b-c72a1dfad361 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:16.322010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09e18c-a94d-49a1-955b-c72a1dfad361 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:16.323288 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09e18c-a94d-49a1-955b-c72a1dfad361 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:16.323938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09e18c-a94d-49a1-955b-c72a1dfad361 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:16.326372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09e18c-a94d-49a1-955b-c72a1dfad361 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:16.332226 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6398c4d-5b8f-40a5-af0c-2fa95f3ff699 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:16.333758 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6398c4d-5b8f-40a5-af0c-2fa95f3ff699 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:16.350390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 367/753] 10.222.0.85 () {56 vars in 1133 bytes} [Mon Aug 29 20:06:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:16.351817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09e18c-a94d-49a1-955b-c72a1dfad361 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:16.462648 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 387/754] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:06:16.463674 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6398c4d-5b8f-40a5-af0c-2fa95f3ff699 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:16.502619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09e18c-a94d-49a1-955b-c72a1dfad361 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:16.551136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140aaf8-3f54-463e-be88-b3390ab90402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:16.552645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140aaf8-3f54-463e-be88-b3390ab90402 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:16.553216 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140aaf8-3f54-463e-be88-b3390ab90402 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:16.553748 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140aaf8-3f54-463e-be88-b3390ab90402 None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:16.554855 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140aaf8-3f54-463e-be88-b3390ab90402 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:16.555319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140aaf8-3f54-463e-be88-b3390ab90402 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:16.557751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140aaf8-3f54-463e-be88-b3390ab90402 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:16.560276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:16.563978 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:16.576379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 368/755] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:16.577610 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140aaf8-3f54-463e-be88-b3390ab90402 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:16.578030 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6398c4d-5b8f-40a5-af0c-2fa95f3ff699 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:16.631999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930bd1b-2c34-4d02-a35e-517bf1071cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:16.632963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930bd1b-2c34-4d02-a35e-517bf1071cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:16.633400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930bd1b-2c34-4d02-a35e-517bf1071cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:16.634287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930bd1b-2c34-4d02-a35e-517bf1071cf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:16.649908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930bd1b-2c34-4d02-a35e-517bf1071cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:16.652561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930bd1b-2c34-4d02-a35e-517bf1071cf9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-0930bd1b-2c34-4d02-a35e-517bf1071cf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:16.674785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:16.676059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:16.677210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:16.677210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:16.677210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:16.679430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 388/756] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:16.680669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930bd1b-2c34-4d02-a35e-517bf1071cf9 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:16.715260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140aaf8-3f54-463e-be88-b3390ab90402 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:16.768381 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92943a9a-5f8d-4ec4-b6f3-efa48509aff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:16.769361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92943a9a-5f8d-4ec4-b6f3-efa48509aff2 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:16.769800 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92943a9a-5f8d-4ec4-b6f3-efa48509aff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:16.770255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92943a9a-5f8d-4ec4-b6f3-efa48509aff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:16.786652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92943a9a-5f8d-4ec4-b6f3-efa48509aff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:16.788195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92943a9a-5f8d-4ec4-b6f3-efa48509aff2 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-92943a9a-5f8d-4ec4-b6f3-efa48509aff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:16.805010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:16.806222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:16.807351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:16.807351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:16.807351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:16.809270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930bd1b-2c34-4d02-a35e-517bf1071cf9 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:16.813655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 369/757] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:16.814674 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92943a9a-5f8d-4ec4-b6f3-efa48509aff2 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:16.862355 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5bdb5-4275-4141-bdcf-c1d6c79cc318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:16.863363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5bdb5-4275-4141-bdcf-c1d6c79cc318 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:16.864188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5bdb5-4275-4141-bdcf-c1d6c79cc318 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:16.864629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5bdb5-4275-4141-bdcf-c1d6c79cc318 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:16.865509 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:16.867398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5bdb5-4275-4141-bdcf-c1d6c79cc318 None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=2d270713965b47fd9dbb2bcbec2634d5) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:16.867887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5bdb5-4275-4141-bdcf-c1d6c79cc318 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:16.870014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5bdb5-4275-4141-bdcf-c1d6c79cc318 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:16.877973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 389/758] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:16 2022] GET /identity/v3/role_assignments?user.id=2d270713965b47fd9dbb2bcbec2634d5&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:16.879677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5bdb5-4275-4141-bdcf-c1d6c79cc318 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:16.885053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:16.929203 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92943a9a-5f8d-4ec4-b6f3-efa48509aff2 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:17.024253 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5bdb5-4275-4141-bdcf-c1d6c79cc318 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:17.045055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ac74a-56e5-4785-bc63-5b88294fd95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:17.045690 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ac74a-56e5-4785-bc63-5b88294fd95b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:17.046654 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ac74a-56e5-4785-bc63-5b88294fd95b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:17.047153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ac74a-56e5-4785-bc63-5b88294fd95b None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:17.047616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ac74a-56e5-4785-bc63-5b88294fd95b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:17.047923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ac74a-56e5-4785-bc63-5b88294fd95b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:17.049351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ac74a-56e5-4785-bc63-5b88294fd95b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:17.068932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 370/759] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:17.074016 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ac74a-56e5-4785-bc63-5b88294fd95b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:17.077051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff88b2c-fefb-402c-8f9f-b0af3eef9c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:17.078114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff88b2c-fefb-402c-8f9f-b0af3eef9c27 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:17.078398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff88b2c-fefb-402c-8f9f-b0af3eef9c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:17.078668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff88b2c-fefb-402c-8f9f-b0af3eef9c27 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:17.079980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff88b2c-fefb-402c-8f9f-b0af3eef9c27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:17.079980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff88b2c-fefb-402c-8f9f-b0af3eef9c27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:17.081010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff88b2c-fefb-402c-8f9f-b0af3eef9c27 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:17.104891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 390/760] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:17.106174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52c116-e24c-453a-b460-d8f68193afbe None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:17.106735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52c116-e24c-453a-b460-d8f68193afbe None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:17.106735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52c116-e24c-453a-b460-d8f68193afbe None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:17.108179 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 391/761] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:17.113655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52c116-e24c-453a-b460-d8f68193afbe None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:17.196637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:17.208159 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ac74a-56e5-4785-bc63-5b88294fd95b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:17.218217 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:17.225369 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52c116-e24c-453a-b460-d8f68193afbe None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:17.247613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef57e4b-d998-440a-bb46-fc08f4fb038f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:17.248629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef57e4b-d998-440a-bb46-fc08f4fb038f None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:17.249130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef57e4b-d998-440a-bb46-fc08f4fb038f None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:17.250195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef57e4b-d998-440a-bb46-fc08f4fb038f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:17.257678 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef57e4b-d998-440a-bb46-fc08f4fb038f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:17.259377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef57e4b-d998-440a-bb46-fc08f4fb038f None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:17.268589 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72027f16-59e4-494c-bd6e-82ca2081b97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:17.269224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72027f16-59e4-494c-bd6e-82ca2081b97e None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:17.269506 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72027f16-59e4-494c-bd6e-82ca2081b97e None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:17.269779 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72027f16-59e4-494c-bd6e-82ca2081b97e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-bef57e4b-d998-440a-bb46-fc08f4fb038f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:17.274292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:17.275452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:17.276378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:17.276378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:17.276378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:17.276378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:17.276378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:17.276378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:17.276378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:17.276378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:17.277266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72027f16-59e4-494c-bd6e-82ca2081b97e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:17.278609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72027f16-59e4-494c-bd6e-82ca2081b97e None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:17.279424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 371/762] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:17.281519 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20909bd1-c591-4ae6-9731-499f8434f9b3 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:17.282038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20909bd1-c591-4ae6-9731-499f8434f9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:17.282525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20909bd1-c591-4ae6-9731-499f8434f9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-72027f16-59e4-494c-bd6e-82ca2081b97e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:17.299597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:17.303510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:17.304412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:17.304412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:17.304412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:17.304412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:17.304412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:17.304412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:17.304412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:17.304412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:17.304412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 392/763] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:17.304412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72027f16-59e4-494c-bd6e-82ca2081b97e None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:17.306551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-20909bd1-c591-4ae6-9731-499f8434f9b3 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:17.308052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-20909bd1-c591-4ae6-9731-499f8434f9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:17.412819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 372/764] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:17.412819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20909bd1-c591-4ae6-9731-499f8434f9b3 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:17.427192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72027f16-59e4-494c-bd6e-82ca2081b97e None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:17.487260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:17.525769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab20ca0-74c1-4749-b08c-e3f2ba208653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:17.526795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab20ca0-74c1-4749-b08c-e3f2ba208653 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:17.527282 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab20ca0-74c1-4749-b08c-e3f2ba208653 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:17.527757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab20ca0-74c1-4749-b08c-e3f2ba208653 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:17.528735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab20ca0-74c1-4749-b08c-e3f2ba208653 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:17.529293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab20ca0-74c1-4749-b08c-e3f2ba208653 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:17.529869 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:17.531596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab20ca0-74c1-4749-b08c-e3f2ba208653 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:17.537737 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20909bd1-c591-4ae6-9731-499f8434f9b3 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:17.559740 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 393/765] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:17.567976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab20ca0-74c1-4749-b08c-e3f2ba208653 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:17.595623 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a254d-5d31-465c-b487-e407bcd8c5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:17.597365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a254d-5d31-465c-b487-e407bcd8c5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:17.599591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a254d-5d31-465c-b487-e407bcd8c5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:17.600028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a254d-5d31-465c-b487-e407bcd8c5f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:17.600926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a254d-5d31-465c-b487-e407bcd8c5f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:17.601420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a254d-5d31-465c-b487-e407bcd8c5f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:17.603478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a254d-5d31-465c-b487-e407bcd8c5f2 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:17.622359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 373/766] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:17.629383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a254d-5d31-465c-b487-e407bcd8c5f2 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:17.747917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab20ca0-74c1-4749-b08c-e3f2ba208653 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:17.776571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a254d-5d31-465c-b487-e407bcd8c5f2 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:17.787776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea1486-f2f4-472a-90ef-dd5393d1b03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:17.789119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea1486-f2f4-472a-90ef-dd5393d1b03b None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:17.790570 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea1486-f2f4-472a-90ef-dd5393d1b03b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:17.791158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea1486-f2f4-472a-90ef-dd5393d1b03b None admin] PATH_INFO: `/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/805df3047fee45c7a702d72a5c082eb5/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:17.793952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea1486-f2f4-472a-90ef-dd5393d1b03b None admin] RBAC: Authorizing `identity:create_grant(project_id=11372fe8994543b4a19773c0e0bee53e, user_id=805df3047fee45c7a702d72a5c082eb5, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:17.795994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea1486-f2f4-472a-90ef-dd5393d1b03b None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:17.811921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0de0f-d526-4f7d-bb15-a2bb80ea24f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:17.816374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0de0f-d526-4f7d-bb15-a2bb80ea24f3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:17.816374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0de0f-d526-4f7d-bb15-a2bb80ea24f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:17.816374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0de0f-d526-4f7d-bb15-a2bb80ea24f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:17.816374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0de0f-d526-4f7d-bb15-a2bb80ea24f3 None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=d52f84245a2142e69c309aa709a7d921) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:17.816374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0de0f-d526-4f7d-bb15-a2bb80ea24f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:17.816374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0de0f-d526-4f7d-bb15-a2bb80ea24f3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:17.820979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 374/767] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:17 2022] GET /identity/v3/role_assignments?user.id=d52f84245a2142e69c309aa709a7d921&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:17.822114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626251db-fa60-4dc2-a5b5-1245882d0b18 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:17.822398 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626251db-fa60-4dc2-a5b5-1245882d0b18 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:17.822669 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626251db-fa60-4dc2-a5b5-1245882d0b18 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:17.824055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 375/768] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:17.824591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626251db-fa60-4dc2-a5b5-1245882d0b18 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:17.856627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:17.888502 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 394/769] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:06:17 2022] PUT /identity/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/805df3047fee45c7a702d72a5c082eb5/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:06:17.890613 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ddf33-4887-4b66-80f1-6376034cd2df None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:17.890851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ddf33-4887-4b66-80f1-6376034cd2df None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:17.891129 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ddf33-4887-4b66-80f1-6376034cd2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:17.932734 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-3b6ddf33-4887-4b66-80f1-6376034cd2df None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:17.934044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3b6ddf33-4887-4b66-80f1-6376034cd2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:17.970354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626251db-fa60-4dc2-a5b5-1245882d0b18 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:18.023831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:18.027470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c3453-ac3f-48f1-8e52-517d0c8493c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:18.028363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c3453-ac3f-48f1-8e52-517d0c8493c1 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:18.028477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c3453-ac3f-48f1-8e52-517d0c8493c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:18.030911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c3453-ac3f-48f1-8e52-517d0c8493c1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:18.030911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c3453-ac3f-48f1-8e52-517d0c8493c1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:18.031224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c3453-ac3f-48f1-8e52-517d0c8493c1 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-cf6c3453-ac3f-48f1-8e52-517d0c8493c1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:18.046374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:18.047450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:18.048339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 376/770] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:17 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:18.056127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c3453-ac3f-48f1-8e52-517d0c8493c1 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:18.110041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 395/771] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:18.161274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:18.173555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c3453-ac3f-48f1-8e52-517d0c8493c1 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:18.219740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaabff-f69c-4797-8f94-ae1f5958df83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:18.220854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaabff-f69c-4797-8f94-ae1f5958df83 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:18.221357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaabff-f69c-4797-8f94-ae1f5958df83 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:18.221849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaabff-f69c-4797-8f94-ae1f5958df83 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:18.222873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaabff-f69c-4797-8f94-ae1f5958df83 None admin] RBAC: Adding query filter params (name=network) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:18.223392 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaabff-f69c-4797-8f94-ae1f5958df83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:18.225045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaabff-f69c-4797-8f94-ae1f5958df83 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:18.230498 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 377/772] 10.222.0.85 () {56 vars in 1112 bytes} [Mon Aug 29 20:06:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:18.235945 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ddf33-4887-4b66-80f1-6376034cd2df None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:18.358717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ddf33-4887-4b66-80f1-6376034cd2df None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:18.393848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd92c0-cad7-4af3-b672-d10705068f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:18.394468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd92c0-cad7-4af3-b672-d10705068f6c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:18.394744 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd92c0-cad7-4af3-b672-d10705068f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:18.395018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd92c0-cad7-4af3-b672-d10705068f6c None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:18.395713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd92c0-cad7-4af3-b672-d10705068f6c None admin] RBAC: Adding query filter params (type=network) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:18.396014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd92c0-cad7-4af3-b672-d10705068f6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:18.397400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd92c0-cad7-4af3-b672-d10705068f6c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:18.403006 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 396/773] 10.222.0.85 () {56 vars in 1112 bytes} [Mon Aug 29 20:06:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:18.408901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaabff-f69c-4797-8f94-ae1f5958df83 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:18.534154 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaabff-f69c-4797-8f94-ae1f5958df83 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:18.537832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd92c0-cad7-4af3-b672-d10705068f6c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:18.593758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb407e43-9ae6-44b3-8690-5e7fc90d67be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:18.594366 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb407e43-9ae6-44b3-8690-5e7fc90d67be None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:18.594693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb407e43-9ae6-44b3-8690-5e7fc90d67be None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:18.594970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb407e43-9ae6-44b3-8690-5e7fc90d67be None admin] PATH_INFO: `/v3/endpoints` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:18.595670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb407e43-9ae6-44b3-8690-5e7fc90d67be None admin] RBAC: Adding query filter params (interface=public, service_id=64e64a04c8914682a69e23b4c15f9cde, region_id=RegionOne) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:18.596726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb407e43-9ae6-44b3-8690-5e7fc90d67be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:18.598087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb407e43-9ae6-44b3-8690-5e7fc90d67be None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:18.617069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 378/774] 10.222.0.85 () {56 vars in 1251 bytes} [Mon Aug 29 20:06:18 2022] GET /identity/v3/endpoints?service_id=64e64a04c8914682a69e23b4c15f9cde&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:18.619399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:18.625205 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb407e43-9ae6-44b3-8690-5e7fc90d67be None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:18.661287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:18.686700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd92c0-cad7-4af3-b672-d10705068f6c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:18.742884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14891bfa-ad4a-49bd-a855-b13087fd4748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:18.743527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14891bfa-ad4a-49bd-a855-b13087fd4748 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:18.743809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14891bfa-ad4a-49bd-a855-b13087fd4748 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:18.744084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14891bfa-ad4a-49bd-a855-b13087fd4748 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:18.744705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14891bfa-ad4a-49bd-a855-b13087fd4748 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:18.746069 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14891bfa-ad4a-49bd-a855-b13087fd4748 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-14891bfa-ad4a-49bd-a855-b13087fd4748 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:18.772812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:18.773890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:06:18.774748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:18.775751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 397/775] 10.222.0.85 () {56 vars in 1129 bytes} [Mon Aug 29 20:06:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:18.782973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14891bfa-ad4a-49bd-a855-b13087fd4748 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:18.823096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb407e43-9ae6-44b3-8690-5e7fc90d67be None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:18.886885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197d539-1c5a-40cf-bbdf-6cdd030a2a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:18.888108 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197d539-1c5a-40cf-bbdf-6cdd030a2a52 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:18.889266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197d539-1c5a-40cf-bbdf-6cdd030a2a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:18.890055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197d539-1c5a-40cf-bbdf-6cdd030a2a52 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:18.892015 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197d539-1c5a-40cf-bbdf-6cdd030a2a52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:18.894276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197d539-1c5a-40cf-bbdf-6cdd030a2a52 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:18.903034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 379/776] 10.222.0.85 () {56 vars in 1087 bytes} [Mon Aug 29 20:06:18 2022] GET /identity/v3/services => generated 772 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:18.927720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0308b7-3be5-477b-a737-cd50186d9b18 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:18.927945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0308b7-3be5-477b-a737-cd50186d9b18 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:18.928292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0308b7-3be5-477b-a737-cd50186d9b18 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:18.929707 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 380/777] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:18.935657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e78e7-c8a7-4258-9a97-a8e99bef64d7 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:18.935975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e78e7-c8a7-4258-9a97-a8e99bef64d7 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:18.936243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e78e7-c8a7-4258-9a97-a8e99bef64d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:18.956996 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-6a6e78e7-c8a7-4258-9a97-a8e99bef64d7 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:18.958232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-6a6e78e7-c8a7-4258-9a97-a8e99bef64d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:18.978225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14891bfa-ad4a-49bd-a855-b13087fd4748 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:18.984540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:19.048701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e225078-f472-4397-aac6-278e12f86fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:19.049334 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e225078-f472-4397-aac6-278e12f86fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:19.049702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e225078-f472-4397-aac6-278e12f86fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:19.049989 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e225078-f472-4397-aac6-278e12f86fb9 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:19.050690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e225078-f472-4397-aac6-278e12f86fb9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:19.050998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e225078-f472-4397-aac6-278e12f86fb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:19.053006 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e225078-f472-4397-aac6-278e12f86fb9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:19.069795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 398/778] 10.222.0.85 () {56 vars in 1124 bytes} [Mon Aug 29 20:06:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:19.071767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e225078-f472-4397-aac6-278e12f86fb9 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:19.095702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 381/779] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:19.189828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e225078-f472-4397-aac6-278e12f86fb9 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:19.226243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b56670-81b4-4022-8e9f-59978198048f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:19.226857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b56670-81b4-4022-8e9f-59978198048f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:19.227137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b56670-81b4-4022-8e9f-59978198048f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:19.227413 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b56670-81b4-4022-8e9f-59978198048f None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:19.228047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b56670-81b4-4022-8e9f-59978198048f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:19.228379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b56670-81b4-4022-8e9f-59978198048f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:19.229749 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b56670-81b4-4022-8e9f-59978198048f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:19.235946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 399/780] 10.222.0.85 () {56 vars in 1124 bytes} [Mon Aug 29 20:06:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:19.241828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e78e7-c8a7-4258-9a97-a8e99bef64d7 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:19.251117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:19.351423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e78e7-c8a7-4258-9a97-a8e99bef64d7 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:19.387072 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b56670-81b4-4022-8e9f-59978198048f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:19.387792 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3aad58-6d51-4ee0-8d74-a608fd7bd0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:19.388455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3aad58-6d51-4ee0-8d74-a608fd7bd0a3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:19.388740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3aad58-6d51-4ee0-8d74-a608fd7bd0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:19.389014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3aad58-6d51-4ee0-8d74-a608fd7bd0a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:19.390291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3aad58-6d51-4ee0-8d74-a608fd7bd0a3 None admin] RBAC: Adding query filter params (interface=public, service_id=92954322711d4eb2b2feb6c41e93b535, region_id=RegionOne) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:19.390613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3aad58-6d51-4ee0-8d74-a608fd7bd0a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:19.391988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3aad58-6d51-4ee0-8d74-a608fd7bd0a3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:19.398596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 382/781] 10.222.0.85 () {56 vars in 1251 bytes} [Mon Aug 29 20:06:19 2022] GET /identity/v3/endpoints?service_id=92954322711d4eb2b2feb6c41e93b535&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:19.408567 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3aad58-6d51-4ee0-8d74-a608fd7bd0a3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:19.436522 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:19.509668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b56670-81b4-4022-8e9f-59978198048f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:19.549596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6d67c-af91-4373-a5a2-3ecec3d8ada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:19.550664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6d67c-af91-4373-a5a2-3ecec3d8ada6 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:19.551151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6d67c-af91-4373-a5a2-3ecec3d8ada6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:19.551770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6d67c-af91-4373-a5a2-3ecec3d8ada6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:19.566365 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6d67c-af91-4373-a5a2-3ecec3d8ada6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-0be6d67c-af91-4373-a5a2-3ecec3d8ada6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:19.573295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:19.574218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:19.575074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:19.575074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:19.575074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:19.575074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:19.575074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:19.575074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:19.575074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:19.575074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:19.575074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:19.575074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:19.575074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:19.575074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:19.578097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 400/782] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:06:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:19.587240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6d67c-af91-4373-a5a2-3ecec3d8ada6 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:19.589415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:19.718816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:19.719021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3aad58-6d51-4ee0-8d74-a608fd7bd0a3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:19.731053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6d67c-af91-4373-a5a2-3ecec3d8ada6 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:19.791078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58e94c-c395-4a11-b9ce-1a9f573ff8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:19.791610 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58e94c-c395-4a11-b9ce-1a9f573ff8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:19.791892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58e94c-c395-4a11-b9ce-1a9f573ff8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:19.792183 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58e94c-c395-4a11-b9ce-1a9f573ff8a3 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:19.792801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58e94c-c395-4a11-b9ce-1a9f573ff8a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:19.794184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58e94c-c395-4a11-b9ce-1a9f573ff8a3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:19.799845 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d30e3d-395e-4471-8028-2cb2c96dd1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:19.800488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d30e3d-395e-4471-8028-2cb2c96dd1bc None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:19.800773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d30e3d-395e-4471-8028-2cb2c96dd1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:19.801045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d30e3d-395e-4471-8028-2cb2c96dd1bc None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:19.801718 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d30e3d-395e-4471-8028-2cb2c96dd1bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:19.802021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d30e3d-395e-4471-8028-2cb2c96dd1bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:19.803428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d30e3d-395e-4471-8028-2cb2c96dd1bc None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:19.809571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 383/783] 10.222.0.85 () {56 vars in 1087 bytes} [Mon Aug 29 20:06:19 2022] GET /identity/v3/services => generated 772 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:19.824291 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 401/784] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:06:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:19.830589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58e94c-c395-4a11-b9ce-1a9f573ff8a3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:19.972123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58e94c-c395-4a11-b9ce-1a9f573ff8a3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:20.025352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcd0f1-d7e0-4028-bdde-c54ba1ba3269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:20.026049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcd0f1-d7e0-4028-bdde-c54ba1ba3269 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:20.026332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcd0f1-d7e0-4028-bdde-c54ba1ba3269 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:20.026607 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcd0f1-d7e0-4028-bdde-c54ba1ba3269 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:20.033091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:20.043838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfcd0f1-d7e0-4028-bdde-c54ba1ba3269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:20.045278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfcd0f1-d7e0-4028-bdde-c54ba1ba3269 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-ddfcd0f1-d7e0-4028-bdde-c54ba1ba3269 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:20.067077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:20.068101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:20.069057 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:20.069057 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:20.069057 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:20.069057 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 384/785] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:20.074094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d30e3d-395e-4471-8028-2cb2c96dd1bc None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:20.123457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9da96-2b6d-4805-84b3-f90f120971aa None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:20.123782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9da96-2b6d-4805-84b3-f90f120971aa None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:20.124003 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9da96-2b6d-4805-84b3-f90f120971aa None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:20.125777 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 385/786] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:20.131313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8b4fa-a4f5-4e51-93db-d0a610f9ebc9 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:20.131916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8b4fa-a4f5-4e51-93db-d0a610f9ebc9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:20.132454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8b4fa-a4f5-4e51-93db-d0a610f9ebc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:20.158776 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-73a8b4fa-a4f5-4e51-93db-d0a610f9ebc9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:20.160390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-73a8b4fa-a4f5-4e51-93db-d0a610f9ebc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:20.216634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d30e3d-395e-4471-8028-2cb2c96dd1bc None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:20.236666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 386/787] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:20.259565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021b318-a199-4d46-b5f8-c5a80ce06ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:20.260278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021b318-a199-4d46-b5f8-c5a80ce06ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:20.260564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021b318-a199-4d46-b5f8-c5a80ce06ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:20.260841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021b318-a199-4d46-b5f8-c5a80ce06ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:20.261632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7021b318-a199-4d46-b5f8-c5a80ce06ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:20.261942 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7021b318-a199-4d46-b5f8-c5a80ce06ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:20.263712 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7021b318-a199-4d46-b5f8-c5a80ce06ed6 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:20.276846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8eb915-e79d-4d54-b81d-015d6bcc1b75 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:20.277496 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8eb915-e79d-4d54-b81d-015d6bcc1b75 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:20.277994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8eb915-e79d-4d54-b81d-015d6bcc1b75 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:20.278273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 402/788] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:20.279955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 387/789] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:20.284272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021b318-a199-4d46-b5f8-c5a80ce06ed6 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:20.286291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efadae-5a67-4a1c-af5c-2dc99a67df50 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:20.286810 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efadae-5a67-4a1c-af5c-2dc99a67df50 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:20.287347 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efadae-5a67-4a1c-af5c-2dc99a67df50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:20.301093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:20.309339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-50efadae-5a67-4a1c-af5c-2dc99a67df50 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:20.310853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-50efadae-5a67-4a1c-af5c-2dc99a67df50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:20.387822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 388/790] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:20.407055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021b318-a199-4d46-b5f8-c5a80ce06ed6 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:20.461831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74acf10-7177-4acb-80eb-7415c0668980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:20.462477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74acf10-7177-4acb-80eb-7415c0668980 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:20.462756 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74acf10-7177-4acb-80eb-7415c0668980 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:20.463029 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74acf10-7177-4acb-80eb-7415c0668980 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:20.473958 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74acf10-7177-4acb-80eb-7415c0668980 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:20.475408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74acf10-7177-4acb-80eb-7415c0668980 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-d74acf10-7177-4acb-80eb-7415c0668980 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:20.490500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:20.491482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:20.492536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:20.492536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:20.492536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:20.492536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:20.492536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:20.492536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:20.492536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:20.492536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:20.492536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:06:20.492536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:20.492536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 403/791] 10.222.0.85 () {56 vars in 1117 bytes} [Mon Aug 29 20:06:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:20.497564 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efadae-5a67-4a1c-af5c-2dc99a67df50 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:20.564693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:20.620079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74acf10-7177-4acb-80eb-7415c0668980 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:20.636639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efadae-5a67-4a1c-af5c-2dc99a67df50 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:20.675118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313494d4-a98f-4200-8ab2-47a4edb7d20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:20.675736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313494d4-a98f-4200-8ab2-47a4edb7d20c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:20.676017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313494d4-a98f-4200-8ab2-47a4edb7d20c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:20.676369 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313494d4-a98f-4200-8ab2-47a4edb7d20c None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:20.677111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313494d4-a98f-4200-8ab2-47a4edb7d20c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:20.677510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313494d4-a98f-4200-8ab2-47a4edb7d20c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:20.678872 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313494d4-a98f-4200-8ab2-47a4edb7d20c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:20.698218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 389/792] 10.222.0.85 () {56 vars in 1149 bytes} [Mon Aug 29 20:06:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:20.705234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313494d4-a98f-4200-8ab2-47a4edb7d20c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:20.742203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:20.776134 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74acf10-7177-4acb-80eb-7415c0668980 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:20.815062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313494d4-a98f-4200-8ab2-47a4edb7d20c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:20.818233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:20.825139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff9782-9b9c-4c01-9da4-9c67f5c03579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:20.828349 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff9782-9b9c-4c01-9da4-9c67f5c03579 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:20.828349 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff9782-9b9c-4c01-9da4-9c67f5c03579 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:20.828349 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff9782-9b9c-4c01-9da4-9c67f5c03579 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:20.842049 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff9782-9b9c-4c01-9da4-9c67f5c03579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-d5ff9782-9b9c-4c01-9da4-9c67f5c03579 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:20.851388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:20.852694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:20.853539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:20.853539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:20.853539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:20.853539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:20.853539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:20.853539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:20.853539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:20.853539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:20.853539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:20.853539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:20.853539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:20.853539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:20.854174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 404/793] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:06:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:20.854243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adefab-38dc-4793-9123-95d21ec8efe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:20.854665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff9782-9b9c-4c01-9da4-9c67f5c03579 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:20.854756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adefab-38dc-4793-9123-95d21ec8efe7 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:20.860726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adefab-38dc-4793-9123-95d21ec8efe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:20.861041 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adefab-38dc-4793-9123-95d21ec8efe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:20.872829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adefab-38dc-4793-9123-95d21ec8efe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:20.874349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adefab-38dc-4793-9123-95d21ec8efe7 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-b9adefab-38dc-4793-9123-95d21ec8efe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:20.890700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:20.891758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:20.892682 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:20.892682 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:20.892682 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:20.892682 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 390/794] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:20.893339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adefab-38dc-4793-9123-95d21ec8efe7 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:20.984272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff9782-9b9c-4c01-9da4-9c67f5c03579 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:21.018556 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:21.024140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e08d0-ca15-4a63-aa07-32afa94ba657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:21.024798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e08d0-ca15-4a63-aa07-32afa94ba657 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.025334 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e08d0-ca15-4a63-aa07-32afa94ba657 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.025608 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e08d0-ca15-4a63-aa07-32afa94ba657 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.026551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adefab-38dc-4793-9123-95d21ec8efe7 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:21.037196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e08d0-ca15-4a63-aa07-32afa94ba657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-265e08d0-ca15-4a63-aa07-32afa94ba657 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:21.042013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:21.042947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:21.043789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:21.043789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:21.043789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:21.043789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:21.043789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:21.043789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:21.043789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:21.043789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:21.043789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:21.043789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:21.043789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:21.043789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:21.043789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 405/795] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:06:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:21.044552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e08d0-ca15-4a63-aa07-32afa94ba657 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:21.068650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff998d6-41ea-48a2-a5d6-71e42cdac19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:21.069629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff998d6-41ea-48a2-a5d6-71e42cdac19b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.070188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff998d6-41ea-48a2-a5d6-71e42cdac19b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.070668 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff998d6-41ea-48a2-a5d6-71e42cdac19b None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.071534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff998d6-41ea-48a2-a5d6-71e42cdac19b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:21.072068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff998d6-41ea-48a2-a5d6-71e42cdac19b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:21.073759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff998d6-41ea-48a2-a5d6-71e42cdac19b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:21.083561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 391/796] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:06:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:21.084712 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff998d6-41ea-48a2-a5d6-71e42cdac19b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:21.111255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:21.159499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e08d0-ca15-4a63-aa07-32afa94ba657 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:21.198056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b7603-84a4-41ec-962d-9c264f6a1d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:21.198619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b7603-84a4-41ec-962d-9c264f6a1d87 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.198900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b7603-84a4-41ec-962d-9c264f6a1d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.199175 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b7603-84a4-41ec-962d-9c264f6a1d87 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.199818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b7603-84a4-41ec-962d-9c264f6a1d87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:21.200249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b7603-84a4-41ec-962d-9c264f6a1d87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:21.201999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b7603-84a4-41ec-962d-9c264f6a1d87 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:21.207996 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff998d6-41ea-48a2-a5d6-71e42cdac19b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:21.215127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 406/797] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:21.216375 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b7603-84a4-41ec-962d-9c264f6a1d87 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:21.244888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169395f-5446-4f54-abd7-9202088017de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:21.245868 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169395f-5446-4f54-abd7-9202088017de None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.246358 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169395f-5446-4f54-abd7-9202088017de None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.246912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169395f-5446-4f54-abd7-9202088017de None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.247893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169395f-5446-4f54-abd7-9202088017de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:21.248459 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169395f-5446-4f54-abd7-9202088017de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:21.250578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169395f-5446-4f54-abd7-9202088017de None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:21.259884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 392/798] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:06:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:21.261191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169395f-5446-4f54-abd7-9202088017de None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:21.276503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:21.331581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b7603-84a4-41ec-962d-9c264f6a1d87 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:21.351249 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:21.370024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadb53b-729c-49bc-8ba4-9402d9ca52c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:21.370969 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadb53b-729c-49bc-8ba4-9402d9ca52c3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.371318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadb53b-729c-49bc-8ba4-9402d9ca52c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.371965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadb53b-729c-49bc-8ba4-9402d9ca52c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.372389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169395f-5446-4f54-abd7-9202088017de None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:21.383009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadb53b-729c-49bc-8ba4-9402d9ca52c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:21.384572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadb53b-729c-49bc-8ba4-9402d9ca52c3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-fbadb53b-729c-49bc-8ba4-9402d9ca52c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:21.400319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:21.401285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:21.402262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:21.402262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:21.402262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:21.402262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 407/799] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:21.403319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadb53b-729c-49bc-8ba4-9402d9ca52c3 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:21.414018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b607b8-d179-407b-af63-22f06660128b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:21.414658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b607b8-d179-407b-af63-22f06660128b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.414932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b607b8-d179-407b-af63-22f06660128b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.415210 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b607b8-d179-407b-af63-22f06660128b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.422739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b607b8-d179-407b-af63-22f06660128b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:21.424338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b607b8-d179-407b-af63-22f06660128b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-c8b607b8-d179-407b-af63-22f06660128b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:21.436978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:21.437955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:21.438846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:21.438846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:21.438846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:21.438846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:21.438846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:21.438846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:21.438846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:21.438846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:21.438846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 393/800] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:21.439979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2174ec-a5b1-4b57-a757-d8f924812108 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.439979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2174ec-a5b1-4b57-a757-d8f924812108 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.439979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2174ec-a5b1-4b57-a757-d8f924812108 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.444672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 394/801] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:21.444672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2174ec-a5b1-4b57-a757-d8f924812108 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:21.523219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadb53b-729c-49bc-8ba4-9402d9ca52c3 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:21.525815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:21.557256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2174ec-a5b1-4b57-a757-d8f924812108 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:21.560586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3b1ca-b5a7-476e-85fb-fe2d12838ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:21.561944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3b1ca-b5a7-476e-85fb-fe2d12838ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.562292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3b1ca-b5a7-476e-85fb-fe2d12838ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.562508 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3b1ca-b5a7-476e-85fb-fe2d12838ce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.573421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3b1ca-b5a7-476e-85fb-fe2d12838ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:21.579373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3b1ca-b5a7-476e-85fb-fe2d12838ce1 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-76c3b1ca-b5a7-476e-85fb-fe2d12838ce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:21.594925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:21.601598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:21.602572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:21.602572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:21.602572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:21.602743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc61199-25e4-4944-aef9-f339210aae7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:21.602743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc61199-25e4-4944-aef9-f339210aae7b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.602743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc61199-25e4-4944-aef9-f339210aae7b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.602954 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 408/802] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:21.603025 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc61199-25e4-4944-aef9-f339210aae7b None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.603093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3b1ca-b5a7-476e-85fb-fe2d12838ce1 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:21.603161 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc61199-25e4-4944-aef9-f339210aae7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:21.603161 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc61199-25e4-4944-aef9-f339210aae7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:21.603161 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc61199-25e4-4944-aef9-f339210aae7b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:21.618144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:21.622417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 395/803] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:21.625907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0321506-f0eb-4677-ba9e-8619f27b691a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.626188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0321506-f0eb-4677-ba9e-8619f27b691a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.626527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0321506-f0eb-4677-ba9e-8619f27b691a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.644026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-a0321506-f0eb-4677-ba9e-8619f27b691a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:21.645250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a0321506-f0eb-4677-ba9e-8619f27b691a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:21.713834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3b1ca-b5a7-476e-85fb-fe2d12838ce1 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:21.719644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 396/804] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:21.720550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0321506-f0eb-4677-ba9e-8619f27b691a None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:21.753543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6746a1-3dda-4b5c-849f-a8838cd2ea11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:21.754163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6746a1-3dda-4b5c-849f-a8838cd2ea11 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.754552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6746a1-3dda-4b5c-849f-a8838cd2ea11 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.754889 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6746a1-3dda-4b5c-849f-a8838cd2ea11 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.755603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6746a1-3dda-4b5c-849f-a8838cd2ea11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:21.755910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6746a1-3dda-4b5c-849f-a8838cd2ea11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:21.757438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6746a1-3dda-4b5c-849f-a8838cd2ea11 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:21.778342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 409/805] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:21.779531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6746a1-3dda-4b5c-849f-a8838cd2ea11 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:21.801243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:21.848467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0321506-f0eb-4677-ba9e-8619f27b691a None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:21.886616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:21.893042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df59a44-951d-4979-8204-08a4184de78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:21.893471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df59a44-951d-4979-8204-08a4184de78a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.893759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df59a44-951d-4979-8204-08a4184de78a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.894035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df59a44-951d-4979-8204-08a4184de78a None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.894684 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df59a44-951d-4979-8204-08a4184de78a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:21.895035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df59a44-951d-4979-8204-08a4184de78a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:21.896400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df59a44-951d-4979-8204-08a4184de78a None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:21.900638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6746a1-3dda-4b5c-849f-a8838cd2ea11 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:21.913881 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 397/806] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:21.913881 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df59a44-951d-4979-8204-08a4184de78a None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:21.947452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13193d6-f3a7-4b74-a7eb-01f1e45323c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:21.948516 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13193d6-f3a7-4b74-a7eb-01f1e45323c0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:21.949032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13193d6-f3a7-4b74-a7eb-01f1e45323c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:21.949504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13193d6-f3a7-4b74-a7eb-01f1e45323c0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:21.957567 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13193d6-f3a7-4b74-a7eb-01f1e45323c0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:21.960464 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13193d6-f3a7-4b74-a7eb-01f1e45323c0 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-c13193d6-f3a7-4b74-a7eb-01f1e45323c0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:21.973990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:21.974995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:21.975927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:21.975927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:21.975927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:21.975927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:21.975927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:21.975927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:21.975927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:21.975927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:21.975927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:06:21.975927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:21.981768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 410/807] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:06:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:21.981768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13193d6-f3a7-4b74-a7eb-01f1e45323c0 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:22.022308 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df59a44-951d-4979-8204-08a4184de78a None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:22.062080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0a82a-dfe1-4ae1-8583-eacd4a0a1c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:22.062767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0a82a-dfe1-4ae1-8583-eacd4a0a1c12 None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:22.063051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0a82a-dfe1-4ae1-8583-eacd4a0a1c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:22.063330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0a82a-dfe1-4ae1-8583-eacd4a0a1c12 None admin] PATH_INFO: `/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/2d270713965b47fd9dbb2bcbec2634d5/roles/cd65aca5e0074973a10adcd1c9a180dc` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:22.080517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0a82a-dfe1-4ae1-8583-eacd4a0a1c12 None admin] RBAC: Authorizing `identity:create_grant(project_id=11372fe8994543b4a19773c0e0bee53e, user_id=2d270713965b47fd9dbb2bcbec2634d5, role_id=cd65aca5e0074973a10adcd1c9a180dc)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:22.082000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0a82a-dfe1-4ae1-8583-eacd4a0a1c12 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:22.172481 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 398/808] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:06:21 2022] PUT /identity/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/2d270713965b47fd9dbb2bcbec2634d5/roles/cd65aca5e0074973a10adcd1c9a180dc => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:06:22.173569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0a82a-dfe1-4ae1-8583-eacd4a0a1c12 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:22.177616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:22.218945 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13193d6-f3a7-4b74-a7eb-01f1e45323c0 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:22.239730 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:22.271727 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0375bfe-9e47-4466-bef6-0a7fe2e3ba61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:22.272620 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0375bfe-9e47-4466-bef6-0a7fe2e3ba61 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:22.273000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0375bfe-9e47-4466-bef6-0a7fe2e3ba61 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:22.273468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0375bfe-9e47-4466-bef6-0a7fe2e3ba61 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:22.285347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0375bfe-9e47-4466-bef6-0a7fe2e3ba61 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:22.288204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0375bfe-9e47-4466-bef6-0a7fe2e3ba61 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-b0375bfe-9e47-4466-bef6-0a7fe2e3ba61 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:22.305491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:22.306605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:22.312714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:22.312714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:22.312714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:22.312714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:22.312714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:22.312714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:22.312714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:22.312714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:22.312714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:06:22.312714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:22.312714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 411/809] 10.222.0.85 () {56 vars in 1093 bytes} [Mon Aug 29 20:06:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:22.312714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0375bfe-9e47-4466-bef6-0a7fe2e3ba61 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:22.317312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0a82a-dfe1-4ae1-8583-eacd4a0a1c12 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:22.359587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f64d1-4698-4256-abfc-f34fe28f03ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:22.360399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f64d1-4698-4256-abfc-f34fe28f03ab None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:22.360680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f64d1-4698-4256-abfc-f34fe28f03ab None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:22.360998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f64d1-4698-4256-abfc-f34fe28f03ab None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:22.361851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f64d1-4698-4256-abfc-f34fe28f03ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:22.362157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f64d1-4698-4256-abfc-f34fe28f03ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:22.363596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f64d1-4698-4256-abfc-f34fe28f03ab None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:22.383922 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 399/810] 10.222.0.85 () {56 vars in 1135 bytes} [Mon Aug 29 20:06:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:22.388817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de895198-31c4-4c8b-9c48-7fd08e2a38a9 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:22.388817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de895198-31c4-4c8b-9c48-7fd08e2a38a9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:22.388817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de895198-31c4-4c8b-9c48-7fd08e2a38a9 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:22.388817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 400/811] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:22.388817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de895198-31c4-4c8b-9c48-7fd08e2a38a9 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:22.426076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0375bfe-9e47-4466-bef6-0a7fe2e3ba61 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:22.462232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb93a9b-b833-4f60-94a6-48fb4b9a3131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:22.462887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb93a9b-b833-4f60-94a6-48fb4b9a3131 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:22.463165 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb93a9b-b833-4f60-94a6-48fb4b9a3131 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:22.463441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb93a9b-b833-4f60-94a6-48fb4b9a3131 None admin] PATH_INFO: `/v3/services/network` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:22.464036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb93a9b-b833-4f60-94a6-48fb4b9a3131 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:22.465474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb93a9b-b833-4f60-94a6-48fb4b9a3131 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:22.469175 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-ceb93a9b-b833-4f60-94a6-48fb4b9a3131 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:22.476000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:22.477428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:06:22.478462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:22.479278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 412/812] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:22 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:22.484208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb93a9b-b833-4f60-94a6-48fb4b9a3131 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:22.591078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb93a9b-b833-4f60-94a6-48fb4b9a3131 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:22.622593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:22.628500 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de895198-31c4-4c8b-9c48-7fd08e2a38a9 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:22.636970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c036b-8cdd-4fa4-b2ef-35f185ffaa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:22.637610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c036b-8cdd-4fa4-b2ef-35f185ffaa13 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:22.637885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c036b-8cdd-4fa4-b2ef-35f185ffaa13 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:22.638156 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c036b-8cdd-4fa4-b2ef-35f185ffaa13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:22.649381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c036b-8cdd-4fa4-b2ef-35f185ffaa13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:22.651083 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c036b-8cdd-4fa4-b2ef-35f185ffaa13 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:22.665260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc70871-6d2a-49a1-b6a2-59bbd58a1355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-9b7c036b-8cdd-4fa4-b2ef-35f185ffaa13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:22.665465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:22.671543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:22.672686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:22.672686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:22.672686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:22.672915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc70871-6d2a-49a1-b6a2-59bbd58a1355 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:22.673002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 413/813] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:22.673084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc70871-6d2a-49a1-b6a2-59bbd58a1355 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:22.673084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc70871-6d2a-49a1-b6a2-59bbd58a1355 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:22.673084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc70871-6d2a-49a1-b6a2-59bbd58a1355 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:22.673304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc341c-26ac-455a-bafe-164e98daac8f None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:22.673387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc70871-6d2a-49a1-b6a2-59bbd58a1355 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:22.673469 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc341c-26ac-455a-bafe-164e98daac8f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:22.673469 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc341c-26ac-455a-bafe-164e98daac8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:22.673626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc70871-6d2a-49a1-b6a2-59bbd58a1355 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:22.690633 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-0ccc341c-26ac-455a-bafe-164e98daac8f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:22.691761 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0ccc341c-26ac-455a-bafe-164e98daac8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:22.697423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 401/814] 10.222.0.85 () {56 vars in 1133 bytes} [Mon Aug 29 20:06:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:22.697423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc70871-6d2a-49a1-b6a2-59bbd58a1355 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:22.752014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:22.777963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 414/815] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:22.777963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc341c-26ac-455a-bafe-164e98daac8f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:22.831975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc70871-6d2a-49a1-b6a2-59bbd58a1355 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:22.899867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c92a9-dc5a-44a2-8b02-bd298cf36063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:22.900801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc341c-26ac-455a-bafe-164e98daac8f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:22.900901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c92a9-dc5a-44a2-8b02-bd298cf36063 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:22.901055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c92a9-dc5a-44a2-8b02-bd298cf36063 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:22.901328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c92a9-dc5a-44a2-8b02-bd298cf36063 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:22.901964 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c92a9-dc5a-44a2-8b02-bd298cf36063 None admin] RBAC: Adding query filter params (name=network) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:22.902267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c92a9-dc5a-44a2-8b02-bd298cf36063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:22.904768 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c92a9-dc5a-44a2-8b02-bd298cf36063 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:22.917439 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 402/816] 10.222.0.85 () {56 vars in 1112 bytes} [Mon Aug 29 20:06:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:22.918461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c92a9-dc5a-44a2-8b02-bd298cf36063 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:22.945406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887b788-556e-4759-857c-d8da8766aa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:22.946012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:22.947085 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887b788-556e-4759-857c-d8da8766aa45 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:22.947636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887b788-556e-4759-857c-d8da8766aa45 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:22.948060 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887b788-556e-4759-857c-d8da8766aa45 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:22.948878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887b788-556e-4759-857c-d8da8766aa45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:22.949330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887b788-556e-4759-857c-d8da8766aa45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:22.950904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887b788-556e-4759-857c-d8da8766aa45 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:22.964200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 415/817] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:22.965044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887b788-556e-4759-857c-d8da8766aa45 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:23.016337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:23.045036 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c92a9-dc5a-44a2-8b02-bd298cf36063 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:23.081310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887b788-556e-4759-857c-d8da8766aa45 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:23.091326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595cacb-d17b-454d-82c4-0af0f4a897ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:23.092058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595cacb-d17b-454d-82c4-0af0f4a897ae None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:23.092470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595cacb-d17b-454d-82c4-0af0f4a897ae None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:23.093368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595cacb-d17b-454d-82c4-0af0f4a897ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:23.120734 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ec829-76cd-4382-be5f-0b78a23f8cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:23.121348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ec829-76cd-4382-be5f-0b78a23f8cfa None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:23.121624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ec829-76cd-4382-be5f-0b78a23f8cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:23.121891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ec829-76cd-4382-be5f-0b78a23f8cfa None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:23.122525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ec829-76cd-4382-be5f-0b78a23f8cfa None admin] RBAC: Adding query filter params (type=network) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:23.122940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ec829-76cd-4382-be5f-0b78a23f8cfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:23.124651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ec829-76cd-4382-be5f-0b78a23f8cfa None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:23.135133 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 416/818] 10.222.0.85 () {56 vars in 1112 bytes} [Mon Aug 29 20:06:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:23.135133 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ec829-76cd-4382-be5f-0b78a23f8cfa None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:23.178867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595cacb-d17b-454d-82c4-0af0f4a897ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:23.180699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595cacb-d17b-454d-82c4-0af0f4a897ae None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:23.206465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:23.239524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ec829-76cd-4382-be5f-0b78a23f8cfa None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:23.246587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:23.279120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5fef3-6b36-4fc1-934c-1b7a6d56717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:23.279867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5fef3-6b36-4fc1-934c-1b7a6d56717a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:23.280262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5fef3-6b36-4fc1-934c-1b7a6d56717a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:23.280628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5fef3-6b36-4fc1-934c-1b7a6d56717a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:23.287493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5fef3-6b36-4fc1-934c-1b7a6d56717a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:23.289425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5fef3-6b36-4fc1-934c-1b7a6d56717a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:23.302484 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 403/819] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:06:23.302484 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595cacb-d17b-454d-82c4-0af0f4a897ae None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-7fe5fef3-6b36-4fc1-934c-1b7a6d56717a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:23.302840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:23.303832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:23.305315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:23.305315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:23.305315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:23.305315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:23.305315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:23.305315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:23.305315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:23.305315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:23.305315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 417/820] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:23.305315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5fef3-6b36-4fc1-934c-1b7a6d56717a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:23.412413 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5fef3-6b36-4fc1-934c-1b7a6d56717a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:23.413085 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595cacb-d17b-454d-82c4-0af0f4a897ae None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:23.433966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:23.447860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1723128-5802-48bc-9293-ba86aacb04f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:23.448516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1723128-5802-48bc-9293-ba86aacb04f9 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:23.448795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1723128-5802-48bc-9293-ba86aacb04f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:23.449058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b329e7-516b-4c48-ac85-6a4100d3ce73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:23.449167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1723128-5802-48bc-9293-ba86aacb04f9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:23.449639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1723128-5802-48bc-9293-ba86aacb04f9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:23.449918 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b329e7-516b-4c48-ac85-6a4100d3ce73 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:23.450297 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b329e7-516b-4c48-ac85-6a4100d3ce73 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:23.450802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b329e7-516b-4c48-ac85-6a4100d3ce73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:23.450998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1723128-5802-48bc-9293-ba86aacb04f9 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:23.451552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b329e7-516b-4c48-ac85-6a4100d3ce73 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:23.453156 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b329e7-516b-4c48-ac85-6a4100d3ce73 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-b1723128-5802-48bc-9293-ba86aacb04f9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:23.460324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:23.461471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:06:23.462451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:23.466354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 404/821] 10.222.0.85 () {56 vars in 1129 bytes} [Mon Aug 29 20:06:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:23.467210 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1723128-5802-48bc-9293-ba86aacb04f9 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:23.566488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1723128-5802-48bc-9293-ba86aacb04f9 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:23.584205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 418/822] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:06:23 2022] POST /identity/v3/endpoints => generated 337 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:06:23.584205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b329e7-516b-4c48-ac85-6a4100d3ce73 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:23.619874 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e3a1b-d5b4-4720-a7a7-0e9f8bcdb8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:23.620797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e3a1b-d5b4-4720-a7a7-0e9f8bcdb8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:23.621171 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e3a1b-d5b4-4720-a7a7-0e9f8bcdb8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:23.621543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e3a1b-d5b4-4720-a7a7-0e9f8bcdb8f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:23.633798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e3a1b-d5b4-4720-a7a7-0e9f8bcdb8f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:23.634873 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:23.635797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e3a1b-d5b4-4720-a7a7-0e9f8bcdb8f7 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-4b9e3a1b-d5b4-4720-a7a7-0e9f8bcdb8f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:23.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:23.654212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:23.655388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:23.655388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:23.655388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:23.660784 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 405/823] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:23.661877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e3a1b-d5b4-4720-a7a7-0e9f8bcdb8f7 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:23.699123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:23.751173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b329e7-516b-4c48-ac85-6a4100d3ce73 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:23.786273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e3a1b-d5b4-4720-a7a7-0e9f8bcdb8f7 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:23.794422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf5886-1251-4cb6-a147-8a37cb1bf1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:23.795045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf5886-1251-4cb6-a147-8a37cb1bf1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:23.795319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf5886-1251-4cb6-a147-8a37cb1bf1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:23.795591 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf5886-1251-4cb6-a147-8a37cb1bf1f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:23.796499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf5886-1251-4cb6-a147-8a37cb1bf1f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:23.796834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf5886-1251-4cb6-a147-8a37cb1bf1f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:23.798322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf5886-1251-4cb6-a147-8a37cb1bf1f3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:23.814818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 419/824] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:23.815505 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf5886-1251-4cb6-a147-8a37cb1bf1f3 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:23.857720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f382a33-8cd2-4888-ad2e-cf5a2f69a8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:23.857720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f382a33-8cd2-4888-ad2e-cf5a2f69a8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:23.857720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f382a33-8cd2-4888-ad2e-cf5a2f69a8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:23.857720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f382a33-8cd2-4888-ad2e-cf5a2f69a8d3 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:23.858199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f382a33-8cd2-4888-ad2e-cf5a2f69a8d3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:23.858511 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f382a33-8cd2-4888-ad2e-cf5a2f69a8d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:23.859891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f382a33-8cd2-4888-ad2e-cf5a2f69a8d3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:23.869762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 406/825] 10.222.0.85 () {56 vars in 1124 bytes} [Mon Aug 29 20:06:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:23.870617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f382a33-8cd2-4888-ad2e-cf5a2f69a8d3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:23.916479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:23.936732 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf5886-1251-4cb6-a147-8a37cb1bf1f3 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:23.974278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:23.979905 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f382a33-8cd2-4888-ad2e-cf5a2f69a8d3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:23.981151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760269c-b734-4da8-98b3-9aa925b1efe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:23.981955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760269c-b734-4da8-98b3-9aa925b1efe5 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:23.982415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760269c-b734-4da8-98b3-9aa925b1efe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:23.982692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760269c-b734-4da8-98b3-9aa925b1efe5 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:23.983380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760269c-b734-4da8-98b3-9aa925b1efe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:23.983678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760269c-b734-4da8-98b3-9aa925b1efe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:23.985064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760269c-b734-4da8-98b3-9aa925b1efe5 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:24.002530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 420/826] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:24.002530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760269c-b734-4da8-98b3-9aa925b1efe5 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:24.017202 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a96439-cda6-415d-9149-1ce2dbaaf289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:24.018361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a96439-cda6-415d-9149-1ce2dbaaf289 None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:24.018795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a96439-cda6-415d-9149-1ce2dbaaf289 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:24.019286 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a96439-cda6-415d-9149-1ce2dbaaf289 None admin] PATH_INFO: `/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/d52f84245a2142e69c309aa709a7d921/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:24.021696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a96439-cda6-415d-9149-1ce2dbaaf289 None admin] RBAC: Authorizing `identity:create_grant(project_id=11372fe8994543b4a19773c0e0bee53e, user_id=d52f84245a2142e69c309aa709a7d921, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:24.023127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a96439-cda6-415d-9149-1ce2dbaaf289 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:24.093859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 407/827] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:06:23 2022] PUT /identity/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/d52f84245a2142e69c309aa709a7d921/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:06:24.095292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a96439-cda6-415d-9149-1ce2dbaaf289 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:24.135192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760269c-b734-4da8-98b3-9aa925b1efe5 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:24.180615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf0d1e-be6d-4ed2-81c7-63a0bab2357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:24.181236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf0d1e-be6d-4ed2-81c7-63a0bab2357f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:24.181514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf0d1e-be6d-4ed2-81c7-63a0bab2357f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:24.182070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf0d1e-be6d-4ed2-81c7-63a0bab2357f None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:24.182428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf0d1e-be6d-4ed2-81c7-63a0bab2357f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:24.182731 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf0d1e-be6d-4ed2-81c7-63a0bab2357f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:24.184143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf0d1e-be6d-4ed2-81c7-63a0bab2357f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:24.193110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 421/828] 10.222.0.85 () {56 vars in 1124 bytes} [Mon Aug 29 20:06:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:24.193110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989a12a-f3ef-498a-85ee-2a0c5aa07fae None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:24.193110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989a12a-f3ef-498a-85ee-2a0c5aa07fae None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:24.193110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989a12a-f3ef-498a-85ee-2a0c5aa07fae None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:24.194705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 422/829] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:24.196363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989a12a-f3ef-498a-85ee-2a0c5aa07fae None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:24.210144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:24.227559 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a96439-cda6-415d-9149-1ce2dbaaf289 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:24.273961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7ad0c-1c59-4a73-82e6-3e7032a8c97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:24.274546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ad0c-1c59-4a73-82e6-3e7032a8c97f None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:24.274837 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ad0c-1c59-4a73-82e6-3e7032a8c97f None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:24.275352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ad0c-1c59-4a73-82e6-3e7032a8c97f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:24.283450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7ad0c-1c59-4a73-82e6-3e7032a8c97f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:24.284948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7ad0c-1c59-4a73-82e6-3e7032a8c97f None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-aaf7ad0c-1c59-4a73-82e6-3e7032a8c97f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:24.297042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:24.298062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:24.298999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:24.298999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:24.298999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:24.298999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:24.298999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:24.298999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:24.298999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:24.298999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:24.298999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 408/830] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:24.299526 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999fcf0-0f69-4759-bfca-f55a5195355d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:24.300272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999fcf0-0f69-4759-bfca-f55a5195355d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:24.300272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999fcf0-0f69-4759-bfca-f55a5195355d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:24.320467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-0999fcf0-0f69-4759-bfca-f55a5195355d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:24.321659 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-0999fcf0-0f69-4759-bfca-f55a5195355d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:24.326444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989a12a-f3ef-498a-85ee-2a0c5aa07fae None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:24.333627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:24.364991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778c41d-e90b-475b-8fad-aa8aaee4a668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:24.365608 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778c41d-e90b-475b-8fad-aa8aaee4a668 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:24.365892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778c41d-e90b-475b-8fad-aa8aaee4a668 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:24.366168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778c41d-e90b-475b-8fad-aa8aaee4a668 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:24.366964 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778c41d-e90b-475b-8fad-aa8aaee4a668 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:24.368377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778c41d-e90b-475b-8fad-aa8aaee4a668 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:24.395155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 409/831] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:24.396088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999fcf0-0f69-4759-bfca-f55a5195355d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:24.496905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 423/832] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:06:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:06:24.510410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999fcf0-0f69-4759-bfca-f55a5195355d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:24.558550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881e0e4-4c9a-4800-b0fd-c30908e16ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:24.559139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881e0e4-4c9a-4800-b0fd-c30908e16ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:24.559421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881e0e4-4c9a-4800-b0fd-c30908e16ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:24.559740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881e0e4-4c9a-4800-b0fd-c30908e16ca3 None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:24.560485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881e0e4-4c9a-4800-b0fd-c30908e16ca3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:24.560792 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881e0e4-4c9a-4800-b0fd-c30908e16ca3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:24.562436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881e0e4-4c9a-4800-b0fd-c30908e16ca3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:24.576121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 410/833] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:24.583008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778c41d-e90b-475b-8fad-aa8aaee4a668 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:24.657667 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:24.680716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881e0e4-4c9a-4800-b0fd-c30908e16ca3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:24.684447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:24.702784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778c41d-e90b-475b-8fad-aa8aaee4a668 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:24.743638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f67cbf-5b7f-4cab-ae54-70dbba567afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:24.744287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f67cbf-5b7f-4cab-ae54-70dbba567afc None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:24.744571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f67cbf-5b7f-4cab-ae54-70dbba567afc None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:24.744847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f67cbf-5b7f-4cab-ae54-70dbba567afc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:24.745581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f67cbf-5b7f-4cab-ae54-70dbba567afc None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=805df3047fee45c7a702d72a5c082eb5) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:24.745895 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f67cbf-5b7f-4cab-ae54-70dbba567afc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:24.747507 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f67cbf-5b7f-4cab-ae54-70dbba567afc None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:24.755612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 424/834] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:24 2022] GET /identity/v3/role_assignments?user.id=805df3047fee45c7a702d72a5c082eb5&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:24.821451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881e0e4-4c9a-4800-b0fd-c30908e16ca3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:24.869982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ae771-e1f7-4d8b-b2d1-85e11029299e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:24.871000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ae771-e1f7-4d8b-b2d1-85e11029299e None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:24.871483 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ae771-e1f7-4d8b-b2d1-85e11029299e None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:24.872061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ae771-e1f7-4d8b-b2d1-85e11029299e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:24.885033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ae771-e1f7-4d8b-b2d1-85e11029299e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-375ae771-e1f7-4d8b-b2d1-85e11029299e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:24.892732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:24.893760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:24.894591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:24.894591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:24.894591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:24.894591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:24.894591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:24.894591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:24.894591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:24.894591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:24.894591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:24.894591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:24.894591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:06:24.894591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:24.900793 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 411/835] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:06:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:24.904095 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f67cbf-5b7f-4cab-ae54-70dbba567afc None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:25.015156 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f67cbf-5b7f-4cab-ae54-70dbba567afc None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:25.042912 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:25.050897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b9159-51b9-4472-973b-60598b6b2f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:25.051561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b9159-51b9-4472-973b-60598b6b2f0a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:25.051840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b9159-51b9-4472-973b-60598b6b2f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:25.052152 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b9159-51b9-4472-973b-60598b6b2f0a None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:25.052922 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b9159-51b9-4472-973b-60598b6b2f0a None admin] RBAC: Adding query filter params (name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:25.053164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b9159-51b9-4472-973b-60598b6b2f0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:25.054530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b9159-51b9-4472-973b-60598b6b2f0a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:25.068982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 425/836] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:06:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:25.071752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ae771-e1f7-4d8b-b2d1-85e11029299e None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:25.141485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:25.183367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ae771-e1f7-4d8b-b2d1-85e11029299e None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:25.227938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cb29a-b093-48b6-8f94-2c53594298f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:25.228597 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cb29a-b093-48b6-8f94-2c53594298f4 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:25.228870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cb29a-b093-48b6-8f94-2c53594298f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:25.229134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cb29a-b093-48b6-8f94-2c53594298f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:25.241127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cb29a-b093-48b6-8f94-2c53594298f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:25.242567 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cb29a-b093-48b6-8f94-2c53594298f4 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-584cb29a-b093-48b6-8f94-2c53594298f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:25.257407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:25.258376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:25.259255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:25.259255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:25.259255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:25.264039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 412/837] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:25.265905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b9159-51b9-4472-973b-60598b6b2f0a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:25.369092 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b9159-51b9-4472-973b-60598b6b2f0a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:25.404116 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3329376-1e15-46f3-9574-c125c694dac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:25.404750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3329376-1e15-46f3-9574-c125c694dac3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:25.405047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3329376-1e15-46f3-9574-c125c694dac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:25.405400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3329376-1e15-46f3-9574-c125c694dac3 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:25.406033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3329376-1e15-46f3-9574-c125c694dac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:25.406331 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3329376-1e15-46f3-9574-c125c694dac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:25.407687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3329376-1e15-46f3-9574-c125c694dac3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:25.424276 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 426/838] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:25.425868 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cb29a-b093-48b6-8f94-2c53594298f4 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:25.530797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cb29a-b093-48b6-8f94-2c53594298f4 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:25.545099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:25.566510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cb4d8-3c76-40f6-93df-69e5cf464ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:25.567243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cb4d8-3c76-40f6-93df-69e5cf464ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:25.567518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cb4d8-3c76-40f6-93df-69e5cf464ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:25.567786 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cb4d8-3c76-40f6-93df-69e5cf464ab6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:25.575737 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cb4d8-3c76-40f6-93df-69e5cf464ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:25.577299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cb4d8-3c76-40f6-93df-69e5cf464ab6 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-859cb4d8-3c76-40f6-93df-69e5cf464ab6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:25.589056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:25.591278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:25.594134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:25.594134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:25.594134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:25.594134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:25.594134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:25.594134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:25.594134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:25.594134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:25.594134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:06:25.594134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:25.600206 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 413/839] 10.222.0.85 () {56 vars in 1117 bytes} [Mon Aug 29 20:06:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:25.600345 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3329376-1e15-46f3-9574-c125c694dac3 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:25.658358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:25.711117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3329376-1e15-46f3-9574-c125c694dac3 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:25.752295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b8030-2b67-4064-9968-2dd37daa0627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:25.753146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b8030-2b67-4064-9968-2dd37daa0627 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:25.753569 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b8030-2b67-4064-9968-2dd37daa0627 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:25.754014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b8030-2b67-4064-9968-2dd37daa0627 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:25.754934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b8030-2b67-4064-9968-2dd37daa0627 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:25.755384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b8030-2b67-4064-9968-2dd37daa0627 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:25.756962 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b8030-2b67-4064-9968-2dd37daa0627 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:25.780301 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 427/840] 10.222.0.85 () {56 vars in 1149 bytes} [Mon Aug 29 20:06:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:25.782861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cb4d8-3c76-40f6-93df-69e5cf464ab6 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:25.885992 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cb4d8-3c76-40f6-93df-69e5cf464ab6 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:25.923247 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2260d-0689-490e-b50f-ba1948a25302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:25.924098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2260d-0689-490e-b50f-ba1948a25302 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:25.924542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2260d-0689-490e-b50f-ba1948a25302 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:25.924982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2260d-0689-490e-b50f-ba1948a25302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:25.976622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:26.002308 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2260d-0689-490e-b50f-ba1948a25302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:26.003815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2260d-0689-490e-b50f-ba1948a25302 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:26.091198 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0310337-e6fc-47ee-b94d-8d547380ba92 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.091533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0310337-e6fc-47ee-b94d-8d547380ba92 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.091746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0310337-e6fc-47ee-b94d-8d547380ba92 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.094106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 428/841] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:26.099347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9198f-c2af-4a44-8760-a388b9b0756e None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.099743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9198f-c2af-4a44-8760-a388b9b0756e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.100110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9198f-c2af-4a44-8760-a388b9b0756e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.111456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 414/842] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:25 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:06:26.120881 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2260d-0689-490e-b50f-ba1948a25302 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:26.127426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-0eb9198f-c2af-4a44-8760-a388b9b0756e None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:26.128748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0eb9198f-c2af-4a44-8760-a388b9b0756e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:26.216281 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 429/843] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:26.228226 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:26.254940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2260d-0689-490e-b50f-ba1948a25302 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:26.292834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665fa4d-b7ec-486b-9577-a873d190a017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:26.293804 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665fa4d-b7ec-486b-9577-a873d190a017 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.294235 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665fa4d-b7ec-486b-9577-a873d190a017 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.294684 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665fa4d-b7ec-486b-9577-a873d190a017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.305790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665fa4d-b7ec-486b-9577-a873d190a017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:26.307490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665fa4d-b7ec-486b-9577-a873d190a017 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-8665fa4d-b7ec-486b-9577-a873d190a017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:26.323507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:26.324847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:26.326248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:26.326248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:26.326248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:26.328569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 415/844] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:26.336200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9198f-c2af-4a44-8760-a388b9b0756e None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:26.378170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:26.451807 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9198f-c2af-4a44-8760-a388b9b0756e None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:26.488663 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbe7d6-61f1-446d-8ba9-85f06b54d9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:26.489543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbe7d6-61f1-446d-8ba9-85f06b54d9c8 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.489990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbe7d6-61f1-446d-8ba9-85f06b54d9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.490415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbe7d6-61f1-446d-8ba9-85f06b54d9c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.491204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbe7d6-61f1-446d-8ba9-85f06b54d9c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:26.491656 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbe7d6-61f1-446d-8ba9-85f06b54d9c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:26.493196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbe7d6-61f1-446d-8ba9-85f06b54d9c8 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:26.506476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 430/845] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:26.513316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665fa4d-b7ec-486b-9577-a873d190a017 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:26.542739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbe7d6-61f1-446d-8ba9-85f06b54d9c8 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:26.629058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665fa4d-b7ec-486b-9577-a873d190a017 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:26.653616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:26.658510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:26.659388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbe7d6-61f1-446d-8ba9-85f06b54d9c8 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:26.668828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ee705-9892-414b-881b-f6c095e97b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:26.670087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ee705-9892-414b-881b-f6c095e97b57 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.671397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ee705-9892-414b-881b-f6c095e97b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.671859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ee705-9892-414b-881b-f6c095e97b57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.680798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ee705-9892-414b-881b-f6c095e97b57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:26.682711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ee705-9892-414b-881b-f6c095e97b57 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:26.694396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508926b-ae74-46d1-9d50-b3f4c37ef3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:26.695028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508926b-ae74-46d1-9d50-b3f4c37ef3bf None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.695311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508926b-ae74-46d1-9d50-b3f4c37ef3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.695580 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508926b-ae74-46d1-9d50-b3f4c37ef3bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-eb6ee705-9892-414b-881b-f6c095e97b57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:26.698835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:26.700994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:26.702119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:26.702119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:26.702119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:26.702119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:26.702119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:26.702119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:26.702119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:26.702119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:26.702962 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 416/846] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:26.704513 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716096ce-c9d6-4801-979e-561c3bfbc4f7 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.704806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716096ce-c9d6-4801-979e-561c3bfbc4f7 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.705084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716096ce-c9d6-4801-979e-561c3bfbc4f7 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.710974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 417/847] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:26.710974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716096ce-c9d6-4801-979e-561c3bfbc4f7 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:26.715616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508926b-ae74-46d1-9d50-b3f4c37ef3bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-d508926b-ae74-46d1-9d50-b3f4c37ef3bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:26.721130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:26.722258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:26.723443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:26.723443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:26.723443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:26.723443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:26.723443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:26.723443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:26.723443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:26.723443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:26.723443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:26.723443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:26.723443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:26.723443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:26.723443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 431/848] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:06:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:26.724120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd15543-0855-4726-a804-74e55125fd73 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.724120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd15543-0855-4726-a804-74e55125fd73 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.724295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd15543-0855-4726-a804-74e55125fd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.743692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-3cd15543-0855-4726-a804-74e55125fd73 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:26.744882 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3cd15543-0855-4726-a804-74e55125fd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:26.828192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 432/849] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:26.828192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd15543-0855-4726-a804-74e55125fd73 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:26.832476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716096ce-c9d6-4801-979e-561c3bfbc4f7 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:26.871908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe37455-6654-471c-bb2b-de70120c0b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:26.872550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe37455-6654-471c-bb2b-de70120c0b14 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.872834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe37455-6654-471c-bb2b-de70120c0b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.873105 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe37455-6654-471c-bb2b-de70120c0b14 None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.874369 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe37455-6654-471c-bb2b-de70120c0b14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:26.874689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe37455-6654-471c-bb2b-de70120c0b14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:26.876288 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe37455-6654-471c-bb2b-de70120c0b14 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:26.888851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 418/850] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:26.892212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e0a28-814f-4cc3-acb2-e636546b3e8a None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.892212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e0a28-814f-4cc3-acb2-e636546b3e8a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.892212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e0a28-814f-4cc3-acb2-e636546b3e8a None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.892212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 419/851] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:26.896539 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e0a28-814f-4cc3-acb2-e636546b3e8a None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:26.928072 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:26.943045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd15543-0855-4726-a804-74e55125fd73 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:26.953488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:26.981974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae825a55-c071-4b4e-91a4-3ad1e70daedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:26.983170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae825a55-c071-4b4e-91a4-3ad1e70daedb None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:26.983457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae825a55-c071-4b4e-91a4-3ad1e70daedb None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:26.983733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae825a55-c071-4b4e-91a4-3ad1e70daedb None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:26.984481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae825a55-c071-4b4e-91a4-3ad1e70daedb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:26.984838 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae825a55-c071-4b4e-91a4-3ad1e70daedb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:26.986831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae825a55-c071-4b4e-91a4-3ad1e70daedb None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:27.002776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 433/852] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:06:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:27.002776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e6732-e570-4541-a324-652e0786503d None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:27.002776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e6732-e570-4541-a324-652e0786503d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:27.002776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e6732-e570-4541-a324-652e0786503d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:27.011924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e0a28-814f-4cc3-acb2-e636546b3e8a None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:27.025715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-5e0e6732-e570-4541-a324-652e0786503d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:27.027172 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5e0e6732-e570-4541-a324-652e0786503d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:27.048485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc3a36-4bfe-46c3-9790-59c8c19397bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:27.049329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc3a36-4bfe-46c3-9790-59c8c19397bb None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:27.049756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc3a36-4bfe-46c3-9790-59c8c19397bb None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:27.050188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc3a36-4bfe-46c3-9790-59c8c19397bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:27.051046 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc3a36-4bfe-46c3-9790-59c8c19397bb None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=2d270713965b47fd9dbb2bcbec2634d5) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:27.051515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc3a36-4bfe-46c3-9790-59c8c19397bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:27.053124 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc3a36-4bfe-46c3-9790-59c8c19397bb None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:27.059290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 420/853] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:26 2022] GET /identity/v3/role_assignments?user.id=2d270713965b47fd9dbb2bcbec2634d5&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:27.060151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc3a36-4bfe-46c3-9790-59c8c19397bb None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:27.128389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 434/854] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:27.128389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e6732-e570-4541-a324-652e0786503d None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:27.195578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc3a36-4bfe-46c3-9790-59c8c19397bb None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:27.212592 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:27.225045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:27.245304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf5505-9d4a-4867-a248-d6bedea168fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:27.245941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf5505-9d4a-4867-a248-d6bedea168fb None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:27.246252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf5505-9d4a-4867-a248-d6bedea168fb None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:27.246507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf5505-9d4a-4867-a248-d6bedea168fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:27.249979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e6732-e570-4541-a324-652e0786503d None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:27.259520 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf5505-9d4a-4867-a248-d6bedea168fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:27.261309 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf5505-9d4a-4867-a248-d6bedea168fb None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-3ecf5505-9d4a-4867-a248-d6bedea168fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:27.277722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:27.279609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:27.280653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:27.280653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:27.280653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:27.280653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 421/855] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:27.284813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf5505-9d4a-4867-a248-d6bedea168fb None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:27.291241 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e45abd-d1a0-4599-893e-ce826ce27f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:27.291893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e45abd-d1a0-4599-893e-ce826ce27f88 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:27.292203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e45abd-d1a0-4599-893e-ce826ce27f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:27.292477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e45abd-d1a0-4599-893e-ce826ce27f88 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:27.293068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e45abd-d1a0-4599-893e-ce826ce27f88 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:27.294431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e45abd-d1a0-4599-893e-ce826ce27f88 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-49e45abd-d1a0-4599-893e-ce826ce27f88 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:27.304673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:27.305745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:27.306587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 435/856] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:06:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:27.312069 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e45abd-d1a0-4599-893e-ce826ce27f88 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:27.419523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf5505-9d4a-4867-a248-d6bedea168fb None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:27.430416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e45abd-d1a0-4599-893e-ce826ce27f88 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:27.456100 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e86b6-d9ea-49e8-8cb6-aa8a4f3205d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:27.456733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e86b6-d9ea-49e8-8cb6-aa8a4f3205d5 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:27.457015 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e86b6-d9ea-49e8-8cb6-aa8a4f3205d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:27.457291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e86b6-d9ea-49e8-8cb6-aa8a4f3205d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:27.458023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e86b6-d9ea-49e8-8cb6-aa8a4f3205d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:27.458329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e86b6-d9ea-49e8-8cb6-aa8a4f3205d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:27.459697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e86b6-d9ea-49e8-8cb6-aa8a4f3205d5 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:27.466053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d60f31-cf8b-4a4a-be21-509813f38297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:27.466658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d60f31-cf8b-4a4a-be21-509813f38297 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:27.466942 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d60f31-cf8b-4a4a-be21-509813f38297 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:27.467307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d60f31-cf8b-4a4a-be21-509813f38297 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:27.467968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d60f31-cf8b-4a4a-be21-509813f38297 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:27.468307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d60f31-cf8b-4a4a-be21-509813f38297 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:27.469664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d60f31-cf8b-4a4a-be21-509813f38297 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:27.472524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 422/857] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:27.474541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 436/858] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:06:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:27.477750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e86b6-d9ea-49e8-8cb6-aa8a4f3205d5 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:27.478075 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d60f31-cf8b-4a4a-be21-509813f38297 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:27.488560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:27.503939 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:27.590389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e86b6-d9ea-49e8-8cb6-aa8a4f3205d5 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:27.591092 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d60f31-cf8b-4a4a-be21-509813f38297 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:27.635553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e2bb5-37ce-496d-90b1-577ed4a075ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:27.636329 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e2bb5-37ce-496d-90b1-577ed4a075ee None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:27.636519 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581049b-12d4-48e0-aac9-283fd2dbd4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:27.636636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e2bb5-37ce-496d-90b1-577ed4a075ee None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:27.636885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e2bb5-37ce-496d-90b1-577ed4a075ee None admin] PATH_INFO: `/v3/users/swift` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:27.637163 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581049b-12d4-48e0-aac9-283fd2dbd4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:27.637497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581049b-12d4-48e0-aac9-283fd2dbd4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:27.637777 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581049b-12d4-48e0-aac9-283fd2dbd4c6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:27.638359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581049b-12d4-48e0-aac9-283fd2dbd4c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:27.640997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581049b-12d4-48e0-aac9-283fd2dbd4c6 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:27.645233 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e2bb5-37ce-496d-90b1-577ed4a075ee None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:27.646673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e2bb5-37ce-496d-90b1-577ed4a075ee None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-e581049b-12d4-48e0-aac9-283fd2dbd4c6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:27.650558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:27.651479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:27.652447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 423/859] 10.222.0.85 () {56 vars in 1132 bytes} [Mon Aug 29 20:06:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:27.656682 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581049b-12d4-48e0-aac9-283fd2dbd4c6 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-4f4e2bb5-37ce-496d-90b1-577ed4a075ee None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:27.663455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:27.664706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:27.665714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:27.665714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:27.665714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:27.665714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:27.665714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:27.665714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:27.665714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:27.665714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:27.665714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:06:27.665714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:27.669326 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 437/860] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:06:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:27.670183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e2bb5-37ce-496d-90b1-577ed4a075ee None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:27.739347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:27.750285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:27.765353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581049b-12d4-48e0-aac9-283fd2dbd4c6 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:27.775736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e2bb5-37ce-496d-90b1-577ed4a075ee None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:27.803448 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143b1e2-8099-4ea7-b50c-e09bd2b97aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:27.804062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b1e2-8099-4ea7-b50c-e09bd2b97aef None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:27.804366 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b1e2-8099-4ea7-b50c-e09bd2b97aef None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:27.804640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b1e2-8099-4ea7-b50c-e09bd2b97aef None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:27.805323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b1e2-8099-4ea7-b50c-e09bd2b97aef None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:27.805677 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b1e2-8099-4ea7-b50c-e09bd2b97aef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:27.807044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b1e2-8099-4ea7-b50c-e09bd2b97aef None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:27.812308 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 424/861] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:06:27 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:27.812308 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143b1e2-8099-4ea7-b50c-e09bd2b97aef None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:27.821485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7f1ec-ec67-4d28-98b7-32c11f75b396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:27.822119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7f1ec-ec67-4d28-98b7-32c11f75b396 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:27.822450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7f1ec-ec67-4d28-98b7-32c11f75b396 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:27.822726 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7f1ec-ec67-4d28-98b7-32c11f75b396 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:27.823496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7f1ec-ec67-4d28-98b7-32c11f75b396 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:27.823833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7f1ec-ec67-4d28-98b7-32c11f75b396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:27.825372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7f1ec-ec67-4d28-98b7-32c11f75b396 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:27.831280 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 438/862] 10.222.0.85 () {56 vars in 1126 bytes} [Mon Aug 29 20:06:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:27.836956 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7f1ec-ec67-4d28-98b7-32c11f75b396 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:27.944047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143b1e2-8099-4ea7-b50c-e09bd2b97aef None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:27.955931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7f1ec-ec67-4d28-98b7-32c11f75b396 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:27.988583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:27.997418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca2e84-fc0f-42c1-b0ee-d94c85532506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:27.998002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca2e84-fc0f-42c1-b0ee-d94c85532506 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:27.998275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca2e84-fc0f-42c1-b0ee-d94c85532506 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:27.998583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca2e84-fc0f-42c1-b0ee-d94c85532506 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:27.999898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca2e84-fc0f-42c1-b0ee-d94c85532506 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:28.001103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca2e84-fc0f-42c1-b0ee-d94c85532506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:28.001562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1434d-7f4d-4da8-b666-c37d5517825b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:28.001679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca2e84-fc0f-42c1-b0ee-d94c85532506 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:28.002247 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1434d-7f4d-4da8-b666-c37d5517825b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:28.002690 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1434d-7f4d-4da8-b666-c37d5517825b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:28.003043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1434d-7f4d-4da8-b666-c37d5517825b None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:28.004051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1434d-7f4d-4da8-b666-c37d5517825b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:28.004051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1434d-7f4d-4da8-b666-c37d5517825b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:28.006778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1434d-7f4d-4da8-b666-c37d5517825b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:28.012247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 439/863] 10.222.0.85 () {56 vars in 1126 bytes} [Mon Aug 29 20:06:27 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:28.029613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 425/864] 10.222.0.85 () {56 vars in 1135 bytes} [Mon Aug 29 20:06:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:28.035749 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca2e84-fc0f-42c1-b0ee-d94c85532506 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:28.158549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca2e84-fc0f-42c1-b0ee-d94c85532506 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:28.198239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80bf76b-7735-4bd9-9f62-67495bac9496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:28.198770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80bf76b-7735-4bd9-9f62-67495bac9496 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:28.199048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80bf76b-7735-4bd9-9f62-67495bac9496 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:28.199319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80bf76b-7735-4bd9-9f62-67495bac9496 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:28.263334 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:28.272726 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80bf76b-7735-4bd9-9f62-67495bac9496 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:28.274536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80bf76b-7735-4bd9-9f62-67495bac9496 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:28.382000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 440/865] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:28 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:06:28.387360 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1434d-7f4d-4da8-b666-c37d5517825b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:28.401216 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:28.495727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1434d-7f4d-4da8-b666-c37d5517825b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:28.533231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e077dd-f108-421f-a714-5ecfa1a011cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:28.533862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e077dd-f108-421f-a714-5ecfa1a011cf None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:28.534140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e077dd-f108-421f-a714-5ecfa1a011cf None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:28.534412 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e077dd-f108-421f-a714-5ecfa1a011cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:28.545202 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e077dd-f108-421f-a714-5ecfa1a011cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:28.546699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e077dd-f108-421f-a714-5ecfa1a011cf None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-d2e077dd-f108-421f-a714-5ecfa1a011cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:28.562352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:28.563349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:28.564727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:28.564727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:28.564727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:28.564727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 426/866] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:28.569279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80bf76b-7735-4bd9-9f62-67495bac9496 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:28.679199 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:28.686631 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80bf76b-7735-4bd9-9f62-67495bac9496 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:28.723241 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60fd4a-c357-4ead-ae92-13eb6761a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:28.723851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60fd4a-c357-4ead-ae92-13eb6761a027 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:28.724128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60fd4a-c357-4ead-ae92-13eb6761a027 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:28.724429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60fd4a-c357-4ead-ae92-13eb6761a027 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:28.725068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60fd4a-c357-4ead-ae92-13eb6761a027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:28.725369 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60fd4a-c357-4ead-ae92-13eb6761a027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:28.726722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60fd4a-c357-4ead-ae92-13eb6761a027 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:28.739401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 441/867] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:28.745317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e077dd-f108-421f-a714-5ecfa1a011cf None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:28.809952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:28.860314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e077dd-f108-421f-a714-5ecfa1a011cf None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:28.894896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0ea5c-d0ef-49f1-a64f-f80c04a4d984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:28.895523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0ea5c-d0ef-49f1-a64f-f80c04a4d984 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:28.895800 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0ea5c-d0ef-49f1-a64f-f80c04a4d984 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:28.896071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0ea5c-d0ef-49f1-a64f-f80c04a4d984 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:28.903122 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0ea5c-d0ef-49f1-a64f-f80c04a4d984 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:28.904593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0ea5c-d0ef-49f1-a64f-f80c04a4d984 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-c2e0ea5c-d0ef-49f1-a64f-f80c04a4d984 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:28.917944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:28.918937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:28.919854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:28.919854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:28.919854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:28.919854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:28.919854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:28.919854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:28.919854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:28.919854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:28.919854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 427/868] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:28.926429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60fd4a-c357-4ead-ae92-13eb6761a027 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:29.038407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60fd4a-c357-4ead-ae92-13eb6761a027 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:29.075799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d502951-e6a4-4cea-81a4-067fa700da5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:29.076387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d502951-e6a4-4cea-81a4-067fa700da5f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:29.076673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d502951-e6a4-4cea-81a4-067fa700da5f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:29.077482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d502951-e6a4-4cea-81a4-067fa700da5f None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:29.078215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d502951-e6a4-4cea-81a4-067fa700da5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:29.078521 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d502951-e6a4-4cea-81a4-067fa700da5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:29.080025 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d502951-e6a4-4cea-81a4-067fa700da5f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:29.095828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 442/869] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:29.101775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0ea5c-d0ef-49f1-a64f-f80c04a4d984 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:29.182253 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868b38a-bb7a-43dd-aaf0-37953fc1ee48 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:29.182570 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868b38a-bb7a-43dd-aaf0-37953fc1ee48 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:29.182789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868b38a-bb7a-43dd-aaf0-37953fc1ee48 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:29.184434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 443/870] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:29.191944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e6c73-5313-4182-8465-3800689cee7a None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:29.192677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e6c73-5313-4182-8465-3800689cee7a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:29.193245 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e6c73-5313-4182-8465-3800689cee7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:29.214367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-a74e6c73-5313-4182-8465-3800689cee7a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:29.215906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a74e6c73-5313-4182-8465-3800689cee7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:29.226146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:29.227569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0ea5c-d0ef-49f1-a64f-f80c04a4d984 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:29.266607 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:29.270734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42ef9c-c686-4c3d-ba70-684d4834bd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:29.271340 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42ef9c-c686-4c3d-ba70-684d4834bd66 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:29.271614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42ef9c-c686-4c3d-ba70-684d4834bd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:29.271893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42ef9c-c686-4c3d-ba70-684d4834bd66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:29.273000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42ef9c-c686-4c3d-ba70-684d4834bd66 None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=d52f84245a2142e69c309aa709a7d921) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:29.273315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42ef9c-c686-4c3d-ba70-684d4834bd66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:29.274745 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42ef9c-c686-4c3d-ba70-684d4834bd66 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:29.284831 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 428/871] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:29 2022] GET /identity/v3/role_assignments?user.id=d52f84245a2142e69c309aa709a7d921&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:29.299464 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 444/872] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:29.572730 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42ef9c-c686-4c3d-ba70-684d4834bd66 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:29.686740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42ef9c-c686-4c3d-ba70-684d4834bd66 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:29.721971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594615b4-13c5-45bb-b3eb-6ab40e795147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:29.722598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594615b4-13c5-45bb-b3eb-6ab40e795147 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:29.722874 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594615b4-13c5-45bb-b3eb-6ab40e795147 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:29.723141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594615b4-13c5-45bb-b3eb-6ab40e795147 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:29.738497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594615b4-13c5-45bb-b3eb-6ab40e795147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-594615b4-13c5-45bb-b3eb-6ab40e795147 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:29.744075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:29.745039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:29.745892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:29.745892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:29.745892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:29.745892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:29.745892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:29.745892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:29.745892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:29.745892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:29.745892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:29.745892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:29.745892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:06:29.745892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:29.748516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 429/873] 10.222.0.85 () {56 vars in 1120 bytes} [Mon Aug 29 20:06:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:29.754401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e6c73-5313-4182-8465-3800689cee7a None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:29.865440 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e6c73-5313-4182-8465-3800689cee7a None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:29.902344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1ccf7-0634-43e8-a53e-9eb2784cb903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:29.903341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1ccf7-0634-43e8-a53e-9eb2784cb903 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:29.903825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1ccf7-0634-43e8-a53e-9eb2784cb903 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:29.904354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1ccf7-0634-43e8-a53e-9eb2784cb903 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:29.905286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1ccf7-0634-43e8-a53e-9eb2784cb903 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:29.905817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1ccf7-0634-43e8-a53e-9eb2784cb903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:29.907433 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1ccf7-0634-43e8-a53e-9eb2784cb903 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:29.916727 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 445/874] 10.222.0.85 () {56 vars in 1115 bytes} [Mon Aug 29 20:06:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:29.925248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594615b4-13c5-45bb-b3eb-6ab40e795147 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:29.977751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:30.011442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499fa86-3a2e-4d84-ad87-401d894fee66 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:30.011784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499fa86-3a2e-4d84-ad87-401d894fee66 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:30.012002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499fa86-3a2e-4d84-ad87-401d894fee66 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:30.018731 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 446/875] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:30.021371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417a528-e6e2-4c6d-b86d-f8a8a687833a None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:30.021654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417a528-e6e2-4c6d-b86d-f8a8a687833a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:30.021927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417a528-e6e2-4c6d-b86d-f8a8a687833a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:30.037583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:30.039634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594615b4-13c5-45bb-b3eb-6ab40e795147 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:30.044562 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-9417a528-e6e2-4c6d-b86d-f8a8a687833a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:30.045955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9417a528-e6e2-4c6d-b86d-f8a8a687833a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:30.077589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e69ae-078c-48af-9a6c-0be5e3e6e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:30.078516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e69ae-078c-48af-9a6c-0be5e3e6e8ac None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:30.078947 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e69ae-078c-48af-9a6c-0be5e3e6e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:30.079368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e69ae-078c-48af-9a6c-0be5e3e6e8ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:30.090716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e69ae-078c-48af-9a6c-0be5e3e6e8ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:30.092218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e69ae-078c-48af-9a6c-0be5e3e6e8ac None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-d04e69ae-078c-48af-9a6c-0be5e3e6e8ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:30.109400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:30.110375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:30.111337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:30.111337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:30.111337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:30.111337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 430/876] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:30.112024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ed65e-c204-4407-b86d-4314c1bfb363 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:30.112297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ed65e-c204-4407-b86d-4314c1bfb363 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:30.112524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ed65e-c204-4407-b86d-4314c1bfb363 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:30.114021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 431/877] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:30.119053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ed65e-c204-4407-b86d-4314c1bfb363 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:30.138465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 447/878] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:30.138465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e240f-ea88-4db7-83a1-d994835a0be8 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:30.138465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e240f-ea88-4db7-83a1-d994835a0be8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:30.138465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e240f-ea88-4db7-83a1-d994835a0be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:30.158813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-5b9e240f-ea88-4db7-83a1-d994835a0be8 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:30.160376 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5b9e240f-ea88-4db7-83a1-d994835a0be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:30.241619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:30.248247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 448/879] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:30.252748 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ed65e-c204-4407-b86d-4314c1bfb363 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:30.289782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b09948-07ea-4447-919b-d94d78fbe4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:30.290446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b09948-07ea-4447-919b-d94d78fbe4cb None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:30.290729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b09948-07ea-4447-919b-d94d78fbe4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:30.291005 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b09948-07ea-4447-919b-d94d78fbe4cb None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:30.291695 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b09948-07ea-4447-919b-d94d78fbe4cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:30.292001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b09948-07ea-4447-919b-d94d78fbe4cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:30.293437 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b09948-07ea-4447-919b-d94d78fbe4cb None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:30.305582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 432/880] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:30.313293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e240f-ea88-4db7-83a1-d994835a0be8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:30.391905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:30.410839 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b09948-07ea-4447-919b-d94d78fbe4cb None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:30.428426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e240f-ea88-4db7-83a1-d994835a0be8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:30.466151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e352c72-f32a-4116-8159-8191cb74270a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:30.466820 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e352c72-f32a-4116-8159-8191cb74270a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:30.467105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e352c72-f32a-4116-8159-8191cb74270a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:30.467443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e352c72-f32a-4116-8159-8191cb74270a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:30.476357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e352c72-f32a-4116-8159-8191cb74270a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:30.477927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e352c72-f32a-4116-8159-8191cb74270a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-6e352c72-f32a-4116-8159-8191cb74270a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:30.491013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:30.492010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:30.493007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:30.493007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:30.493007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:30.493007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:30.493007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:30.493007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:30.493007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:30.493007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:30.493007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:06:30.493007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:30.496646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 449/881] 10.222.0.85 () {56 vars in 1117 bytes} [Mon Aug 29 20:06:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:30.498105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e352c72-f32a-4116-8159-8191cb74270a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:30.519181 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b09948-07ea-4447-919b-d94d78fbe4cb None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:30.555938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca796ffa-1c86-4710-ba6c-247cfacbd1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:30.556688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca796ffa-1c86-4710-ba6c-247cfacbd1cc None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:30.556998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca796ffa-1c86-4710-ba6c-247cfacbd1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:30.557251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca796ffa-1c86-4710-ba6c-247cfacbd1cc None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:30.557940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca796ffa-1c86-4710-ba6c-247cfacbd1cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:30.559388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca796ffa-1c86-4710-ba6c-247cfacbd1cc None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:30.604840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e352c72-f32a-4116-8159-8191cb74270a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:30.630381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:30.638925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a76f64-d40d-4f37-8cba-931d2e4dac65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:30.640072 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a76f64-d40d-4f37-8cba-931d2e4dac65 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:30.640385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a76f64-d40d-4f37-8cba-931d2e4dac65 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:30.640716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a76f64-d40d-4f37-8cba-931d2e4dac65 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:30.641438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a76f64-d40d-4f37-8cba-931d2e4dac65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:30.641679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a76f64-d40d-4f37-8cba-931d2e4dac65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:30.643290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a76f64-d40d-4f37-8cba-931d2e4dac65 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:30.648208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 433/882] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:06:30 2022] POST /identity/v3/services => generated 269 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:06:30.648208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca796ffa-1c86-4710-ba6c-247cfacbd1cc None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:30.672733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 450/883] 10.222.0.85 () {56 vars in 1149 bytes} [Mon Aug 29 20:06:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:30.674157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a76f64-d40d-4f37-8cba-931d2e4dac65 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:30.683282 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:30.802483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a76f64-d40d-4f37-8cba-931d2e4dac65 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:30.822113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca796ffa-1c86-4710-ba6c-247cfacbd1cc None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:30.848386 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff6923-b570-4171-9366-8a2e4906245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:30.849360 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff6923-b570-4171-9366-8a2e4906245a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:30.849950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff6923-b570-4171-9366-8a2e4906245a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:30.850429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff6923-b570-4171-9366-8a2e4906245a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:30.864066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd566f92-c735-4638-ad6e-085914cfd1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:30.864701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd566f92-c735-4638-ad6e-085914cfd1ac None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:30.864984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd566f92-c735-4638-ad6e-085914cfd1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:30.865294 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd566f92-c735-4638-ad6e-085914cfd1ac None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:30.865888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd566f92-c735-4638-ad6e-085914cfd1ac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:30.867249 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd566f92-c735-4638-ad6e-085914cfd1ac None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:30.912031 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:30.930581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff6923-b570-4171-9366-8a2e4906245a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:30.932051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff6923-b570-4171-9366-8a2e4906245a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:30.963378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 434/884] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:06:30 2022] POST /identity/v3/services => generated 251 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:06:31.065437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 451/885] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:30 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:06:31.073378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd566f92-c735-4638-ad6e-085914cfd1ac None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:31.153076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:31.195040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd566f92-c735-4638-ad6e-085914cfd1ac None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:31.230586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d6900-fb4a-4cbe-aefe-cfcf5b0b2e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:31.231223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d6900-fb4a-4cbe-aefe-cfcf5b0b2e1c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:31.231557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d6900-fb4a-4cbe-aefe-cfcf5b0b2e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:31.231836 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d6900-fb4a-4cbe-aefe-cfcf5b0b2e1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:31.244588 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d6900-fb4a-4cbe-aefe-cfcf5b0b2e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:31.246045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d6900-fb4a-4cbe-aefe-cfcf5b0b2e1c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-1c3d6900-fb4a-4cbe-aefe-cfcf5b0b2e1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:31.263487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:31.265248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:31.266324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:31.266324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:31.266324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:31.268052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 435/886] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:31.276425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff6923-b570-4171-9366-8a2e4906245a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:31.347642 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036eba7-7515-455b-87aa-7fc57b4c5726 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:31.348304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036eba7-7515-455b-87aa-7fc57b4c5726 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:31.348789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036eba7-7515-455b-87aa-7fc57b4c5726 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:31.350829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 436/887] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:31.355825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c232f0-cf9a-4bff-9157-1a66410568c6 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:31.356123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c232f0-cf9a-4bff-9157-1a66410568c6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:31.356487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c232f0-cf9a-4bff-9157-1a66410568c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:31.371462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:31.380021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-23c232f0-cf9a-4bff-9157-1a66410568c6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:31.381568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-23c232f0-cf9a-4bff-9157-1a66410568c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:31.388227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff6923-b570-4171-9366-8a2e4906245a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:31.425982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83db63a-4e1a-42f3-a6c5-4ed46602a3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:31.426683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83db63a-4e1a-42f3-a6c5-4ed46602a3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:31.427051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83db63a-4e1a-42f3-a6c5-4ed46602a3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:31.427438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83db63a-4e1a-42f3-a6c5-4ed46602a3e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:31.428147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83db63a-4e1a-42f3-a6c5-4ed46602a3e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:31.428542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83db63a-4e1a-42f3-a6c5-4ed46602a3e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:31.430154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83db63a-4e1a-42f3-a6c5-4ed46602a3e7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:31.442038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 452/888] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:31.453417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83db63a-4e1a-42f3-a6c5-4ed46602a3e7 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:31.472145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 437/889] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:31.559374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83db63a-4e1a-42f3-a6c5-4ed46602a3e7 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:31.594891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed43ec2-2e65-42ad-932d-efe2bb84a5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:31.595525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed43ec2-2e65-42ad-932d-efe2bb84a5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:31.595805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed43ec2-2e65-42ad-932d-efe2bb84a5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:31.596079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed43ec2-2e65-42ad-932d-efe2bb84a5e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:31.604803 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed43ec2-2e65-42ad-932d-efe2bb84a5e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:31.605290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed43ec2-2e65-42ad-932d-efe2bb84a5e7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-9ed43ec2-2e65-42ad-932d-efe2bb84a5e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:31.618320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:31.619200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:31.620149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:31.620149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:31.620149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:31.620149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:31.620149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:31.620149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:31.620149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:31.620149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:31.620149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 453/890] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:31.626040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c232f0-cf9a-4bff-9157-1a66410568c6 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:31.677775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:31.729068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c232f0-cf9a-4bff-9157-1a66410568c6 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:31.746414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed43ec2-2e65-42ad-932d-efe2bb84a5e7 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:31.752724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:31.771695 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acb8c0-9482-444a-9d3d-770523e3f8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:31.775524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acb8c0-9482-444a-9d3d-770523e3f8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:31.775524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acb8c0-9482-444a-9d3d-770523e3f8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:31.775524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acb8c0-9482-444a-9d3d-770523e3f8f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:31.775524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acb8c0-9482-444a-9d3d-770523e3f8f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:31.775524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acb8c0-9482-444a-9d3d-770523e3f8f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:31.775873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acb8c0-9482-444a-9d3d-770523e3f8f3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:31.788979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 438/891] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:31.794598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acb8c0-9482-444a-9d3d-770523e3f8f3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:31.884224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed43ec2-2e65-42ad-932d-efe2bb84a5e7 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:31.913513 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acb8c0-9482-444a-9d3d-770523e3f8f3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:31.926704 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:31.939708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca85ae6-5273-4df7-9802-ddd1d7afedad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:31.947162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca85ae6-5273-4df7-9802-ddd1d7afedad None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:31.947162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca85ae6-5273-4df7-9802-ddd1d7afedad None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:31.947162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca85ae6-5273-4df7-9802-ddd1d7afedad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:31.947162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca85ae6-5273-4df7-9802-ddd1d7afedad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:31.947162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca85ae6-5273-4df7-9802-ddd1d7afedad None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:31.952964 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e2abe-730e-4173-ace8-aed53d7d391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:31.953648 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e2abe-730e-4173-ace8-aed53d7d391d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:31.954003 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e2abe-730e-4173-ace8-aed53d7d391d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:31.954387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e2abe-730e-4173-ace8-aed53d7d391d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:31.955200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e2abe-730e-4173-ace8-aed53d7d391d None admin] RBAC: Adding query filter params (role.id=130698a8e0c6468491e7d81752709939, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=2d270713965b47fd9dbb2bcbec2634d5) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:31.955581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e2abe-730e-4173-ace8-aed53d7d391d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:31.957147 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e2abe-730e-4173-ace8-aed53d7d391d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-0ca85ae6-5273-4df7-9802-ddd1d7afedad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:31.957790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:31.958883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:31.959840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:31.966939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 454/892] 10.222.0.85 () {56 vars in 1126 bytes} [Mon Aug 29 20:06:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:31.969601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca85ae6-5273-4df7-9802-ddd1d7afedad None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:31.971386 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 439/893] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:31 2022] GET /identity/v3/role_assignments?user.id=2d270713965b47fd9dbb2bcbec2634d5&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=130698a8e0c6468491e7d81752709939 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:32.054153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:32.093715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca85ae6-5273-4df7-9802-ddd1d7afedad None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:32.133985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9bb97-7c41-45dd-ab8b-f8980b40a7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:32.134610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9bb97-7c41-45dd-ab8b-f8980b40a7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:32.134918 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9bb97-7c41-45dd-ab8b-f8980b40a7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:32.135194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9bb97-7c41-45dd-ab8b-f8980b40a7c3 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:32.135836 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9bb97-7c41-45dd-ab8b-f8980b40a7c3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:32.136143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9bb97-7c41-45dd-ab8b-f8980b40a7c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:32.137541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9bb97-7c41-45dd-ab8b-f8980b40a7c3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:32.144625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 455/894] 10.222.0.85 () {56 vars in 1122 bytes} [Mon Aug 29 20:06:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:32.150220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e2abe-730e-4173-ace8-aed53d7d391d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:32.258601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e2abe-730e-4173-ace8-aed53d7d391d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:32.294574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef420b-afa4-4967-819e-e4ae05e411ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:32.295239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef420b-afa4-4967-819e-e4ae05e411ee None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:32.295722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef420b-afa4-4967-819e-e4ae05e411ee None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:32.295988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef420b-afa4-4967-819e-e4ae05e411ee None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:32.296660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef420b-afa4-4967-819e-e4ae05e411ee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:32.296964 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef420b-afa4-4967-819e-e4ae05e411ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:32.298341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef420b-afa4-4967-819e-e4ae05e411ee None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:32.303393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 440/895] 10.222.0.85 () {56 vars in 1122 bytes} [Mon Aug 29 20:06:32 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:32.747620 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de7523-3971-4c5f-b492-e3be259002c2 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:32.747972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de7523-3971-4c5f-b492-e3be259002c2 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:32.748308 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de7523-3971-4c5f-b492-e3be259002c2 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:32.750019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 456/896] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:32.755361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866b57e-18d1-467b-99a9-7109c774dec1 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:32.755848 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866b57e-18d1-467b-99a9-7109c774dec1 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:32.755945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866b57e-18d1-467b-99a9-7109c774dec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:32.774689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-9866b57e-18d1-467b-99a9-7109c774dec1 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:32.776151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9866b57e-18d1-467b-99a9-7109c774dec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:32.806762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:32.845477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 441/897] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:33.075023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ee38-55bd-45d9-9735-d013d92c17c4 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:33.075555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ee38-55bd-45d9-9735-d013d92c17c4 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:33.075555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ee38-55bd-45d9-9735-d013d92c17c4 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:33.077377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 457/898] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:33.082608 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b8cf5-07e2-49b3-b237-1783201dad19 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:33.082905 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b8cf5-07e2-49b3-b237-1783201dad19 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:33.083187 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b8cf5-07e2-49b3-b237-1783201dad19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:33.102073 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-5e6b8cf5-07e2-49b3-b237-1783201dad19 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:33.103262 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5e6b8cf5-07e2-49b3-b237-1783201dad19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:33.116538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7ee38-55bd-45d9-9735-d013d92c17c4 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:33.180312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 442/899] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:33.243197 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7ee38-55bd-45d9-9735-d013d92c17c4 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:33.295453 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:33.303564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ec4cc-18d7-456f-83f8-d794e3297292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:33.304648 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ec4cc-18d7-456f-83f8-d794e3297292 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:33.305127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ec4cc-18d7-456f-83f8-d794e3297292 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:33.305591 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ec4cc-18d7-456f-83f8-d794e3297292 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:33.306626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ec4cc-18d7-456f-83f8-d794e3297292 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:33.309139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ec4cc-18d7-456f-83f8-d794e3297292 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-5e6ec4cc-18d7-456f-83f8-d794e3297292 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:33.326206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:33.327247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:06:33.328191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:33.332610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 458/900] 10.222.0.85 () {56 vars in 1132 bytes} [Mon Aug 29 20:06:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:33.337757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b8cf5-07e2-49b3-b237-1783201dad19 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:33.447943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b8cf5-07e2-49b3-b237-1783201dad19 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:33.456512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:33.487562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa42a50-64ba-4aa9-b191-c63db0a7bf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:33.488668 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa42a50-64ba-4aa9-b191-c63db0a7bf25 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:33.489292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa42a50-64ba-4aa9-b191-c63db0a7bf25 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:33.490103 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa42a50-64ba-4aa9-b191-c63db0a7bf25 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:33.490740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa42a50-64ba-4aa9-b191-c63db0a7bf25 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:33.491046 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa42a50-64ba-4aa9-b191-c63db0a7bf25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:33.492444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa42a50-64ba-4aa9-b191-c63db0a7bf25 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:33.497772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 443/901] 10.222.0.85 () {56 vars in 1126 bytes} [Mon Aug 29 20:06:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:33.503052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ec4cc-18d7-456f-83f8-d794e3297292 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:33.528836 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa42a50-64ba-4aa9-b191-c63db0a7bf25 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:33.611432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ec4cc-18d7-456f-83f8-d794e3297292 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:33.638655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa42a50-64ba-4aa9-b191-c63db0a7bf25 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:33.650321 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b9f65-2673-423b-8902-3b21732d03dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:33.651185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b9f65-2673-423b-8902-3b21732d03dc None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:33.651619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b9f65-2673-423b-8902-3b21732d03dc None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:33.652052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b9f65-2673-423b-8902-3b21732d03dc None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:33.652912 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b9f65-2673-423b-8902-3b21732d03dc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:33.653386 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b9f65-2673-423b-8902-3b21732d03dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:33.654990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b9f65-2673-423b-8902-3b21732d03dc None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:33.664302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 459/902] 10.222.0.85 () {56 vars in 1126 bytes} [Mon Aug 29 20:06:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:33.666630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b9f65-2673-423b-8902-3b21732d03dc None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:33.677231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abd055-ed76-4c79-98ea-ff87c4e02381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:33.678329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abd055-ed76-4c79-98ea-ff87c4e02381 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:33.678329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abd055-ed76-4c79-98ea-ff87c4e02381 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:33.678604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abd055-ed76-4c79-98ea-ff87c4e02381 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:33.679188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abd055-ed76-4c79-98ea-ff87c4e02381 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:33.680700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abd055-ed76-4c79-98ea-ff87c4e02381 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-a2abd055-ed76-4c79-98ea-ff87c4e02381 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:33.690304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:33.691387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:06:33.692259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:33.696411 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 444/903] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:06:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:33.697416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abd055-ed76-4c79-98ea-ff87c4e02381 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:33.733227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:33.734021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:33.776693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b9f65-2673-423b-8902-3b21732d03dc None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:33.802312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abd055-ed76-4c79-98ea-ff87c4e02381 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:33.813399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690aa31-6d65-4d32-a214-7c24b99a3762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:33.814280 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690aa31-6d65-4d32-a214-7c24b99a3762 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:33.814732 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690aa31-6d65-4d32-a214-7c24b99a3762 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:33.815199 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690aa31-6d65-4d32-a214-7c24b99a3762 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:33.816093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690aa31-6d65-4d32-a214-7c24b99a3762 None admin] RBAC: Adding query filter params (interface=public, service_id=9babb830ea69458383cb9bddbeb161ed, region_id=RegionOne) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:33.816629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690aa31-6d65-4d32-a214-7c24b99a3762 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:33.818170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690aa31-6d65-4d32-a214-7c24b99a3762 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:33.831714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 460/904] 10.222.0.85 () {56 vars in 1251 bytes} [Mon Aug 29 20:06:33 2022] GET /identity/v3/endpoints?service_id=9babb830ea69458383cb9bddbeb161ed&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:33.832923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690aa31-6d65-4d32-a214-7c24b99a3762 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:33.839959 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06191ec3-61c7-4152-a736-c7c39d641a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:33.840918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06191ec3-61c7-4152-a736-c7c39d641a95 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:33.841274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06191ec3-61c7-4152-a736-c7c39d641a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:33.841619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06191ec3-61c7-4152-a736-c7c39d641a95 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:33.843038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06191ec3-61c7-4152-a736-c7c39d641a95 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:33.843168 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06191ec3-61c7-4152-a736-c7c39d641a95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:33.844658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06191ec3-61c7-4152-a736-c7c39d641a95 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:33.855682 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 445/905] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:06:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:33.857081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06191ec3-61c7-4152-a736-c7c39d641a95 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:33.950055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690aa31-6d65-4d32-a214-7c24b99a3762 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:33.978129 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:33.990128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cddc47-a5b0-45ec-9143-dfa0ce4d67b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:33.991205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cddc47-a5b0-45ec-9143-dfa0ce4d67b6 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:33.991445 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:33.991552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cddc47-a5b0-45ec-9143-dfa0ce4d67b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:33.991886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cddc47-a5b0-45ec-9143-dfa0ce4d67b6 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:33.992622 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cddc47-a5b0-45ec-9143-dfa0ce4d67b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:33.994069 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cddc47-a5b0-45ec-9143-dfa0ce4d67b6 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:33.997207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06191ec3-61c7-4152-a736-c7c39d641a95 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:34.007891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 461/906] 10.222.0.85 () {56 vars in 1087 bytes} [Mon Aug 29 20:06:33 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:06:34.007891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a904-4c32-445f-9531-73217fb6defb None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:34.007891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a904-4c32-445f-9531-73217fb6defb None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:34.007891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a904-4c32-445f-9531-73217fb6defb None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:34.007891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 462/907] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:34.014172 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196e8fc-38a4-4039-a4b2-55b3e92e364a None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:34.014476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196e8fc-38a4-4039-a4b2-55b3e92e364a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:34.014702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196e8fc-38a4-4039-a4b2-55b3e92e364a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:34.035952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7196e8fc-38a4-4039-a4b2-55b3e92e364a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:34.037328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7196e8fc-38a4-4039-a4b2-55b3e92e364a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:34.047471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa6ccf-b6fa-4ec7-a875-25db6d6d0b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:34.048487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa6ccf-b6fa-4ec7-a875-25db6d6d0b45 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:34.049472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa6ccf-b6fa-4ec7-a875-25db6d6d0b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:34.049954 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa6ccf-b6fa-4ec7-a875-25db6d6d0b45 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:34.051199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa6ccf-b6fa-4ec7-a875-25db6d6d0b45 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:34.052061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa6ccf-b6fa-4ec7-a875-25db6d6d0b45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:34.053945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa6ccf-b6fa-4ec7-a875-25db6d6d0b45 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:34.062167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 446/908] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:06:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:34.067727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa6ccf-b6fa-4ec7-a875-25db6d6d0b45 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:34.134847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 463/909] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:34.197726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa6ccf-b6fa-4ec7-a875-25db6d6d0b45 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:34.234349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193a630-fe35-434f-af86-c13909ba83d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:34.235075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193a630-fe35-434f-af86-c13909ba83d3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:34.235355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193a630-fe35-434f-af86-c13909ba83d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:34.235625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193a630-fe35-434f-af86-c13909ba83d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:34.236354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193a630-fe35-434f-af86-c13909ba83d3 None admin] RBAC: Adding query filter params (interface=public, service_id=3558eb720b5a4010a8cc37e8bc68564b, region_id=RegionOne) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:34.236662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193a630-fe35-434f-af86-c13909ba83d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:34.238024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193a630-fe35-434f-af86-c13909ba83d3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:34.243778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 447/910] 10.222.0.85 () {56 vars in 1251 bytes} [Mon Aug 29 20:06:34 2022] GET /identity/v3/endpoints?service_id=3558eb720b5a4010a8cc37e8bc68564b&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:34.249222 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196e8fc-38a4-4039-a4b2-55b3e92e364a None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:34.355083 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196e8fc-38a4-4039-a4b2-55b3e92e364a None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:34.368066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb500e6a-05bf-4796-b634-13ffb63e9245 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:34.368412 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb500e6a-05bf-4796-b634-13ffb63e9245 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:34.368661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb500e6a-05bf-4796-b634-13ffb63e9245 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:34.370271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 448/911] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:34.374749 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:34.375379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97982ee8-d92c-49e3-afb0-bd67eaf2edc4 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:34.375677 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97982ee8-d92c-49e3-afb0-bd67eaf2edc4 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:34.375945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97982ee8-d92c-49e3-afb0-bd67eaf2edc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:34.393107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb7458-f896-4afb-bf5d-daa23ab3aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:34.393723 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb7458-f896-4afb-bf5d-daa23ab3aa55 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:34.395481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb7458-f896-4afb-bf5d-daa23ab3aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:34.395481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb7458-f896-4afb-bf5d-daa23ab3aa55 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:34.395481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb7458-f896-4afb-bf5d-daa23ab3aa55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:34.395715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-97982ee8-d92c-49e3-afb0-bd67eaf2edc4 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:34.396284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb7458-f896-4afb-bf5d-daa23ab3aa55 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:34.396866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-97982ee8-d92c-49e3-afb0-bd67eaf2edc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:34.402883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 464/912] 10.222.0.85 () {56 vars in 1087 bytes} [Mon Aug 29 20:06:34 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:06:34.412148 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:34.477274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 449/913] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:34.548045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb7458-f896-4afb-bf5d-daa23ab3aa55 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:34.696360 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb7458-f896-4afb-bf5d-daa23ab3aa55 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:34.731353 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1854d-1adc-416b-b694-f8850a0fb1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:34.731990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1854d-1adc-416b-b694-f8850a0fb1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:34.732309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1854d-1adc-416b-b694-f8850a0fb1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:34.732586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1854d-1adc-416b-b694-f8850a0fb1d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:34.743059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1854d-1adc-416b-b694-f8850a0fb1d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-f8a1854d-1adc-416b-b694-f8850a0fb1d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:34.747908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:34.749564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:34.750415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:34.750415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:34.750415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:34.750415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:34.750415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:34.750415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:34.750415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:34.750415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:34.750415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:34.750415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:34.750415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:06:34.750415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:34.750415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 465/914] 10.222.0.85 () {56 vars in 1120 bytes} [Mon Aug 29 20:06:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:34.755628 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97982ee8-d92c-49e3-afb0-bd67eaf2edc4 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:34.859675 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97982ee8-d92c-49e3-afb0-bd67eaf2edc4 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:34.866804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1854d-1adc-416b-b694-f8850a0fb1d2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:34.900419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782492ba-85bc-4c6b-8661-185b7afb4317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:34.901029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782492ba-85bc-4c6b-8661-185b7afb4317 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:34.901427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782492ba-85bc-4c6b-8661-185b7afb4317 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:34.901712 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782492ba-85bc-4c6b-8661-185b7afb4317 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:34.902715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782492ba-85bc-4c6b-8661-185b7afb4317 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:34.903120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782492ba-85bc-4c6b-8661-185b7afb4317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:34.904632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782492ba-85bc-4c6b-8661-185b7afb4317 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:34.914626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 450/915] 10.222.0.85 () {56 vars in 1115 bytes} [Mon Aug 29 20:06:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:34.922899 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782492ba-85bc-4c6b-8661-185b7afb4317 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:34.927019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:34.935715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:34.989655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1854d-1adc-416b-b694-f8850a0fb1d2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:35.033613 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfef760-f02b-43ed-9032-8bdc04fcb3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:35.034495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfef760-f02b-43ed-9032-8bdc04fcb3f0 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:35.034819 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfef760-f02b-43ed-9032-8bdc04fcb3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:35.035121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfef760-f02b-43ed-9032-8bdc04fcb3f0 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:35.035835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfef760-f02b-43ed-9032-8bdc04fcb3f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:35.038234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfef760-f02b-43ed-9032-8bdc04fcb3f0 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:35.046736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782492ba-85bc-4c6b-8661-185b7afb4317 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:35.083169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e29d65-2b4d-40d5-93d2-a2e24844c716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:35.083806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e29d65-2b4d-40d5-93d2-a2e24844c716 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:35.084082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e29d65-2b4d-40d5-93d2-a2e24844c716 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:35.084396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e29d65-2b4d-40d5-93d2-a2e24844c716 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:35.095372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e29d65-2b4d-40d5-93d2-a2e24844c716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:35.096855 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e29d65-2b4d-40d5-93d2-a2e24844c716 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-c3e29d65-2b4d-40d5-93d2-a2e24844c716 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:35.112199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:35.113184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:35.114072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:35.114072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:35.114072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:35.117968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 451/916] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:35.119262 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e29d65-2b4d-40d5-93d2-a2e24844c716 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:35.128438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 466/917] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:06:34 2022] POST /identity/v3/services => generated 241 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:06:35.200375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:35.246733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e29d65-2b4d-40d5-93d2-a2e24844c716 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:35.292115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9cb63-1dc7-404f-bdc8-2c16bc8d0a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:35.293437 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9cb63-1dc7-404f-bdc8-2c16bc8d0a47 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:35.294140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9cb63-1dc7-404f-bdc8-2c16bc8d0a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:35.294877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9cb63-1dc7-404f-bdc8-2c16bc8d0a47 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:35.296261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9cb63-1dc7-404f-bdc8-2c16bc8d0a47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:35.297070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9cb63-1dc7-404f-bdc8-2c16bc8d0a47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:35.299447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9cb63-1dc7-404f-bdc8-2c16bc8d0a47 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:35.314192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 452/918] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:35.320067 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfef760-f02b-43ed-9032-8bdc04fcb3f0 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:35.432449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfef760-f02b-43ed-9032-8bdc04fcb3f0 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:35.434397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:35.469269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d46dd61-5744-4587-b575-fa2e012c88c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:35.470276 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46dd61-5744-4587-b575-fa2e012c88c9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:35.470762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46dd61-5744-4587-b575-fa2e012c88c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:35.471230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46dd61-5744-4587-b575-fa2e012c88c9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:35.479763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d46dd61-5744-4587-b575-fa2e012c88c9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d46dd61-5744-4587-b575-fa2e012c88c9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-8d46dd61-5744-4587-b575-fa2e012c88c9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:35.496638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:35.497454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:35.499010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 467/919] 10.222.0.85 () {56 vars in 1117 bytes} [Mon Aug 29 20:06:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:35.501277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9cb63-1dc7-404f-bdc8-2c16bc8d0a47 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:35.615748 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9cb63-1dc7-404f-bdc8-2c16bc8d0a47 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:35.636972 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:35.651907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f4ad6-6996-4c17-823c-526bda26e314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:35.652656 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f4ad6-6996-4c17-823c-526bda26e314 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:35.652974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f4ad6-6996-4c17-823c-526bda26e314 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:35.653228 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f4ad6-6996-4c17-823c-526bda26e314 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:35.654031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f4ad6-6996-4c17-823c-526bda26e314 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:35.654340 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f4ad6-6996-4c17-823c-526bda26e314 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:35.655842 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f4ad6-6996-4c17-823c-526bda26e314 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:35.679450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 453/920] 10.222.0.85 () {56 vars in 1149 bytes} [Mon Aug 29 20:06:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:35.680970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d46dd61-5744-4587-b575-fa2e012c88c9 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:35.788484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d46dd61-5744-4587-b575-fa2e012c88c9 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:35.823270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015302e6-8297-4fd5-b3be-57ad5546e3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:35.823897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015302e6-8297-4fd5-b3be-57ad5546e3f1 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:35.824191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015302e6-8297-4fd5-b3be-57ad5546e3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:35.824491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015302e6-8297-4fd5-b3be-57ad5546e3f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:35.836015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015302e6-8297-4fd5-b3be-57ad5546e3f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:35.837691 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015302e6-8297-4fd5-b3be-57ad5546e3f1 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-015302e6-8297-4fd5-b3be-57ad5546e3f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:35.852513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:35.853515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:35.854789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:35.854789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:35.854789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:35.854789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 468/921] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:35.861748 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f4ad6-6996-4c17-823c-526bda26e314 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:35.967255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f4ad6-6996-4c17-823c-526bda26e314 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:36.003829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee3a71-7eb5-41ad-ab68-cf132f24d285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:36.004866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee3a71-7eb5-41ad-ab68-cf132f24d285 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:36.005350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee3a71-7eb5-41ad-ab68-cf132f24d285 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:36.005825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee3a71-7eb5-41ad-ab68-cf132f24d285 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:36.006668 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee3a71-7eb5-41ad-ab68-cf132f24d285 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:36.007193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee3a71-7eb5-41ad-ab68-cf132f24d285 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:36.008794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee3a71-7eb5-41ad-ab68-cf132f24d285 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:36.026231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 454/922] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:36.029263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015302e6-8297-4fd5-b3be-57ad5546e3f1 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:36.051169 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:36.051581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3292cdd-3383-46d5-bd8b-763e175a96f4 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:36.051873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3292cdd-3383-46d5-bd8b-763e175a96f4 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:36.052220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3292cdd-3383-46d5-bd8b-763e175a96f4 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:36.053805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 455/923] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:36.058741 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d98c3-6e96-4a50-9dc0-9d37f9648408 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:36.059014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d98c3-6e96-4a50-9dc0-9d37f9648408 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:36.059292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d98c3-6e96-4a50-9dc0-9d37f9648408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:36.080726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-616d98c3-6e96-4a50-9dc0-9d37f9648408 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:36.082362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-616d98c3-6e96-4a50-9dc0-9d37f9648408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:36.118697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:36.157058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015302e6-8297-4fd5-b3be-57ad5546e3f1 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:36.173240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 456/924] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:36.194140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fd572-e5fc-4dc2-a9a3-83994588377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:36.195171 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fd572-e5fc-4dc2-a9a3-83994588377f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:36.195654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fd572-e5fc-4dc2-a9a3-83994588377f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:36.196145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fd572-e5fc-4dc2-a9a3-83994588377f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:36.204020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fd572-e5fc-4dc2-a9a3-83994588377f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:36.205773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fd572-e5fc-4dc2-a9a3-83994588377f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-629fd572-e5fc-4dc2-a9a3-83994588377f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:36.218258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:36.219238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:36.220121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:36.220121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:36.220121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:36.220121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:36.220121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:36.220121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:36.220121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:36.220121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:36.227138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 469/925] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:36.229274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d98c3-6e96-4a50-9dc0-9d37f9648408 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:36.340871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d98c3-6e96-4a50-9dc0-9d37f9648408 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:36.385390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc85fdd-dd3a-4c6a-8774-aedfe677915d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:36.386009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc85fdd-dd3a-4c6a-8774-aedfe677915d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:36.386292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc85fdd-dd3a-4c6a-8774-aedfe677915d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:36.386567 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc85fdd-dd3a-4c6a-8774-aedfe677915d None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:36.387278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc85fdd-dd3a-4c6a-8774-aedfe677915d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:36.387587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc85fdd-dd3a-4c6a-8774-aedfe677915d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:36.389557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc85fdd-dd3a-4c6a-8774-aedfe677915d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:36.402593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 457/926] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:36.409388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fd572-e5fc-4dc2-a9a3-83994588377f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:36.453246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc85fdd-dd3a-4c6a-8774-aedfe677915d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:36.486763 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:36.497544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:36.523570 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fd572-e5fc-4dc2-a9a3-83994588377f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:36.559529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d757c6-c510-43e4-bf63-2a1de0144fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:36.560208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d757c6-c510-43e4-bf63-2a1de0144fc9 None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:36.560497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d757c6-c510-43e4-bf63-2a1de0144fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:36.560768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d757c6-c510-43e4-bf63-2a1de0144fc9 None admin] PATH_INFO: `/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/2d270713965b47fd9dbb2bcbec2634d5/roles/130698a8e0c6468491e7d81752709939` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:36.563007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d757c6-c510-43e4-bf63-2a1de0144fc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=11372fe8994543b4a19773c0e0bee53e, user_id=2d270713965b47fd9dbb2bcbec2634d5, role_id=130698a8e0c6468491e7d81752709939)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:36.564490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d757c6-c510-43e4-bf63-2a1de0144fc9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:36.567586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc85fdd-dd3a-4c6a-8774-aedfe677915d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:36.607231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20430aa5-dc79-4b7a-ba52-e9565b1951a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:36.608320 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20430aa5-dc79-4b7a-ba52-e9565b1951a8 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:36.608570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20430aa5-dc79-4b7a-ba52-e9565b1951a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:36.609174 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20430aa5-dc79-4b7a-ba52-e9565b1951a8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:36.609771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20430aa5-dc79-4b7a-ba52-e9565b1951a8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:36.611169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20430aa5-dc79-4b7a-ba52-e9565b1951a8 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-20430aa5-dc79-4b7a-ba52-e9565b1951a8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:36.621655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:36.626705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:36.627649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 458/927] 10.222.0.85 () {56 vars in 1132 bytes} [Mon Aug 29 20:06:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:36.628570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b204829-49d7-42ba-919c-275194e86668 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:36.628570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b204829-49d7-42ba-919c-275194e86668 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:36.628570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b204829-49d7-42ba-919c-275194e86668 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:36.628570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 459/928] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:36.630678 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b204829-49d7-42ba-919c-275194e86668 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:36.676204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 470/929] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:06:36 2022] PUT /identity/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/2d270713965b47fd9dbb2bcbec2634d5/roles/130698a8e0c6468491e7d81752709939 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:06:36.676204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66bd44-e90a-4b5b-83d9-0968009a1c10 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:36.676204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66bd44-e90a-4b5b-83d9-0968009a1c10 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:36.676204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66bd44-e90a-4b5b-83d9-0968009a1c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:36.694106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-af66bd44-e90a-4b5b-83d9-0968009a1c10 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:36.695459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-af66bd44-e90a-4b5b-83d9-0968009a1c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:36.756707 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:36.762340 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b204829-49d7-42ba-919c-275194e86668 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:36.796194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 471/930] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:36.808148 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e9816-9795-460f-80a4-23266a324f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:36.808819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e9816-9795-460f-80a4-23266a324f7b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:36.809101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e9816-9795-460f-80a4-23266a324f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:36.809430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e9816-9795-460f-80a4-23266a324f7b None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:36.810072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e9816-9795-460f-80a4-23266a324f7b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:36.810429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e9816-9795-460f-80a4-23266a324f7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:36.812433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e9816-9795-460f-80a4-23266a324f7b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:36.818624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 460/931] 10.222.0.85 () {56 vars in 1126 bytes} [Mon Aug 29 20:06:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:36.826816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66bd44-e90a-4b5b-83d9-0968009a1c10 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:36.894182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:36.933831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66bd44-e90a-4b5b-83d9-0968009a1c10 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:36.974684 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb814d-affa-4bdc-a14b-0751e4b9eb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:36.998162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb814d-affa-4bdc-a14b-0751e4b9eb85 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:36.998162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb814d-affa-4bdc-a14b-0751e4b9eb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:36.998162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb814d-affa-4bdc-a14b-0751e4b9eb85 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:36.998162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb814d-affa-4bdc-a14b-0751e4b9eb85 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:36.998162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb814d-affa-4bdc-a14b-0751e4b9eb85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:36.998162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb814d-affa-4bdc-a14b-0751e4b9eb85 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:37.007983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 472/932] 10.222.0.85 () {56 vars in 1126 bytes} [Mon Aug 29 20:06:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:37.008072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e9816-9795-460f-80a4-23266a324f7b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:37.077137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb814d-affa-4bdc-a14b-0751e4b9eb85 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:37.108588 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e9816-9795-460f-80a4-23266a324f7b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:37.145498 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed014d19-5dbb-4cd5-907b-618ccaf913b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:37.145983 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed014d19-5dbb-4cd5-907b-618ccaf913b0 None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:37.146258 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed014d19-5dbb-4cd5-907b-618ccaf913b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:37.146530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed014d19-5dbb-4cd5-907b-618ccaf913b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:37.147199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed014d19-5dbb-4cd5-907b-618ccaf913b0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:37.148846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed014d19-5dbb-4cd5-907b-618ccaf913b0 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:37.197034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb814d-affa-4bdc-a14b-0751e4b9eb85 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:37.223444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:37.231589 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e51c2-9a31-4587-b6b8-7cd8e10aa5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:37.232379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e51c2-9a31-4587-b6b8-7cd8e10aa5fb None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:37.233313 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e51c2-9a31-4587-b6b8-7cd8e10aa5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:37.233787 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e51c2-9a31-4587-b6b8-7cd8e10aa5fb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:37.234851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e51c2-9a31-4587-b6b8-7cd8e10aa5fb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:37.237158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e51c2-9a31-4587-b6b8-7cd8e10aa5fb None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:37.240391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 461/933] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:06:36 2022] POST /identity/v3/endpoints => generated 347 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:06:37.240391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e75c2-bf2c-489f-aaf3-430bac7fe318 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:37.240391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e75c2-bf2c-489f-aaf3-430bac7fe318 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:37.240391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e75c2-bf2c-489f-aaf3-430bac7fe318 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:37.240854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 462/934] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:37.246456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf298d63-6ae3-4e44-86ba-3f5d27caf39d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:37.246766 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf298d63-6ae3-4e44-86ba-3f5d27caf39d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:37.247015 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf298d63-6ae3-4e44-86ba-3f5d27caf39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-df2e51c2-9a31-4587-b6b8-7cd8e10aa5fb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:37.250602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:37.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:37.253294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 473/935] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:06:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:37.258210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e51c2-9a31-4587-b6b8-7cd8e10aa5fb None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:37.272444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-bf298d63-6ae3-4e44-86ba-3f5d27caf39d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:37.274039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bf298d63-6ae3-4e44-86ba-3f5d27caf39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:37.277752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:37.378004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 463/936] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:37.393284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e51c2-9a31-4587-b6b8-7cd8e10aa5fb None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:37.433309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396d4df-f5f6-40cd-9252-ad8be4f30b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:37.433970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396d4df-f5f6-40cd-9252-ad8be4f30b86 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:37.434245 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396d4df-f5f6-40cd-9252-ad8be4f30b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:37.434462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396d4df-f5f6-40cd-9252-ad8be4f30b86 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:37.435095 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396d4df-f5f6-40cd-9252-ad8be4f30b86 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:37.435398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396d4df-f5f6-40cd-9252-ad8be4f30b86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:37.436832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396d4df-f5f6-40cd-9252-ad8be4f30b86 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:37.442170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 474/937] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:06:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:37.447752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf298d63-6ae3-4e44-86ba-3f5d27caf39d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:37.550562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf298d63-6ae3-4e44-86ba-3f5d27caf39d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:37.585536 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dc74f-c295-41fb-8f07-06342a9ae9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:37.586152 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dc74f-c295-41fb-8f07-06342a9ae9bf None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:37.586430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dc74f-c295-41fb-8f07-06342a9ae9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:37.586701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dc74f-c295-41fb-8f07-06342a9ae9bf None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:37.587333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8dc74f-c295-41fb-8f07-06342a9ae9bf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:37.587635 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8dc74f-c295-41fb-8f07-06342a9ae9bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:37.589028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8dc74f-c295-41fb-8f07-06342a9ae9bf None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:37.594744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 464/938] 10.222.0.85 () {56 vars in 1114 bytes} [Mon Aug 29 20:06:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:37.595315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:37.600330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396d4df-f5f6-40cd-9252-ad8be4f30b86 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:37.660179 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dc74f-c295-41fb-8f07-06342a9ae9bf None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:37.661658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:37.713858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396d4df-f5f6-40cd-9252-ad8be4f30b86 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:37.762916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efea83-9db3-4798-861c-893d25b12b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:37.763879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efea83-9db3-4798-861c-893d25b12b64 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:37.764244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efea83-9db3-4798-861c-893d25b12b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:37.764666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efea83-9db3-4798-861c-893d25b12b64 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:37.765489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efea83-9db3-4798-861c-893d25b12b64 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:37.767068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efea83-9db3-4798-861c-893d25b12b64 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:37.770171 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dc74f-c295-41fb-8f07-06342a9ae9bf None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:37.809474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2683b-4253-4261-9657-defec2af6ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:37.810112 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2683b-4253-4261-9657-defec2af6ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:37.810390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2683b-4253-4261-9657-defec2af6ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:37.810661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2683b-4253-4261-9657-defec2af6ff4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:37.811859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2683b-4253-4261-9657-defec2af6ff4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:37.813251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2683b-4253-4261-9657-defec2af6ff4 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-e4e2683b-4253-4261-9657-defec2af6ff4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:37.823438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:37.825180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:37.826061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 465/939] 10.222.0.85 () {56 vars in 1126 bytes} [Mon Aug 29 20:06:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:37.830638 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2683b-4253-4261-9657-defec2af6ff4 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:37.905065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 475/940] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:06:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:06:37.912291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:37.961542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2683b-4253-4261-9657-defec2af6ff4 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:38.016718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f078b-270a-4535-b181-4825091a05d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:38.018166 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f078b-270a-4535-b181-4825091a05d2 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:38.018667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f078b-270a-4535-b181-4825091a05d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:38.019145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f078b-270a-4535-b181-4825091a05d2 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:38.020001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f078b-270a-4535-b181-4825091a05d2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:38.020567 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f078b-270a-4535-b181-4825091a05d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:38.022190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f078b-270a-4535-b181-4825091a05d2 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:38.028776 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 466/941] 10.222.0.85 () {56 vars in 1122 bytes} [Mon Aug 29 20:06:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:38.035480 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efea83-9db3-4798-861c-893d25b12b64 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:38.146141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efea83-9db3-4798-861c-893d25b12b64 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:38.161192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:38.187001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f30c7-d312-4391-9e4b-1b1d28195aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:38.187996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f30c7-d312-4391-9e4b-1b1d28195aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:38.188510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f30c7-d312-4391-9e4b-1b1d28195aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:38.189003 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f30c7-d312-4391-9e4b-1b1d28195aa8 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:38.189847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f30c7-d312-4391-9e4b-1b1d28195aa8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:38.190356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f30c7-d312-4391-9e4b-1b1d28195aa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:38.192009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f30c7-d312-4391-9e4b-1b1d28195aa8 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:38.197796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 476/942] 10.222.0.85 () {56 vars in 1122 bytes} [Mon Aug 29 20:06:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:38.204719 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f078b-270a-4535-b181-4825091a05d2 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:38.316708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f078b-270a-4535-b181-4825091a05d2 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:38.342670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:38.353525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8843d6f-530b-46d8-b037-bca253cb89bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:38.354140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8843d6f-530b-46d8-b037-bca253cb89bf None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:38.354416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8843d6f-530b-46d8-b037-bca253cb89bf None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:38.354742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8843d6f-530b-46d8-b037-bca253cb89bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:38.355482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8843d6f-530b-46d8-b037-bca253cb89bf None admin] RBAC: Adding query filter params (interface=public, service_id=c732d9974c3a4e80bc12b9315cb2653a, region_id=RegionOne) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:38.355783 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8843d6f-530b-46d8-b037-bca253cb89bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:38.357170 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8843d6f-530b-46d8-b037-bca253cb89bf None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:38.363080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 467/943] 10.222.0.85 () {56 vars in 1251 bytes} [Mon Aug 29 20:06:38 2022] GET /identity/v3/endpoints?service_id=c732d9974c3a4e80bc12b9315cb2653a&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:38.369475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f30c7-d312-4391-9e4b-1b1d28195aa8 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:38.487781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f30c7-d312-4391-9e4b-1b1d28195aa8 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:38.522335 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfe9ab-40fb-4758-bb85-8945b30453a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:38.522994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfe9ab-40fb-4758-bb85-8945b30453a7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:38.523279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfe9ab-40fb-4758-bb85-8945b30453a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:38.523554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfe9ab-40fb-4758-bb85-8945b30453a7 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:38.524160 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfe9ab-40fb-4758-bb85-8945b30453a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:38.525790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfe9ab-40fb-4758-bb85-8945b30453a7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:38.532711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 477/944] 10.222.0.85 () {56 vars in 1087 bytes} [Mon Aug 29 20:06:38 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:06:38.746665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bdb09-aa9d-49de-8658-f814bb48d120 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:38.746950 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bdb09-aa9d-49de-8658-f814bb48d120 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:38.747222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bdb09-aa9d-49de-8658-f814bb48d120 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:38.748923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 468/945] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:38.754021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753684d-8d26-482c-adfa-d83db0052bd2 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:38.754309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753684d-8d26-482c-adfa-d83db0052bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:38.754583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753684d-8d26-482c-adfa-d83db0052bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:38.774054 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-0753684d-8d26-482c-adfa-d83db0052bd2 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:38.775501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0753684d-8d26-482c-adfa-d83db0052bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:38.812148 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:38.848112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 478/946] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:39.121271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bdb09-aa9d-49de-8658-f814bb48d120 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:39.233734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bdb09-aa9d-49de-8658-f814bb48d120 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:39.268638 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57073db5-7b9c-4f7c-a6e9-86e00b181e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:39.269275 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57073db5-7b9c-4f7c-a6e9-86e00b181e89 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:39.269557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57073db5-7b9c-4f7c-a6e9-86e00b181e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:39.269826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57073db5-7b9c-4f7c-a6e9-86e00b181e89 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:39.281950 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57073db5-7b9c-4f7c-a6e9-86e00b181e89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-57073db5-7b9c-4f7c-a6e9-86e00b181e89 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:39.286755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:39.287667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:39.288551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:39.288551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:39.288551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:39.288551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:39.288551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:39.288551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:39.288551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:39.288551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:39.288551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:39.288551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:39.288551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:06:39.288551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:39.288551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 469/947] 10.222.0.85 () {56 vars in 1120 bytes} [Mon Aug 29 20:06:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:39.293701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753684d-8d26-482c-adfa-d83db0052bd2 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:39.314831 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34745c-d513-4bce-9c88-e212209f0bda None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:39.315160 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34745c-d513-4bce-9c88-e212209f0bda None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:39.315382 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34745c-d513-4bce-9c88-e212209f0bda None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:39.316980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 470/948] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:39.321720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb057e-0fe4-4a92-9e0d-763422288c44 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:39.322005 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb057e-0fe4-4a92-9e0d-763422288c44 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:39.322285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb057e-0fe4-4a92-9e0d-763422288c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:39.340981 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-44cb057e-0fe4-4a92-9e0d-763422288c44 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:39.342707 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-44cb057e-0fe4-4a92-9e0d-763422288c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:39.389547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:39.404590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753684d-8d26-482c-adfa-d83db0052bd2 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:39.420860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 471/949] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:39.443004 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca6682-0c28-473f-8122-fd19709cecfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:39.443829 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca6682-0c28-473f-8122-fd19709cecfd None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:39.444284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca6682-0c28-473f-8122-fd19709cecfd None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:39.444750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca6682-0c28-473f-8122-fd19709cecfd None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:39.445546 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca6682-0c28-473f-8122-fd19709cecfd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:39.446065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca6682-0c28-473f-8122-fd19709cecfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:39.447616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca6682-0c28-473f-8122-fd19709cecfd None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:39.457286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 479/950] 10.222.0.85 () {56 vars in 1115 bytes} [Mon Aug 29 20:06:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:39.463083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb057e-0fe4-4a92-9e0d-763422288c44 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:39.568929 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb057e-0fe4-4a92-9e0d-763422288c44 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:39.604737 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb2f6a-84cd-4c39-bcdd-85d1d3bda713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:39.605450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb2f6a-84cd-4c39-bcdd-85d1d3bda713 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:39.605746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb2f6a-84cd-4c39-bcdd-85d1d3bda713 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:39.606001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb2f6a-84cd-4c39-bcdd-85d1d3bda713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:39.620190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb2f6a-84cd-4c39-bcdd-85d1d3bda713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:39.620190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb2f6a-84cd-4c39-bcdd-85d1d3bda713 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-24eb2f6a-84cd-4c39-bcdd-85d1d3bda713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:39.634416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:39.635463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:39.636386 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:39.636386 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:39.636386 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:39.636386 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 472/951] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:39.642192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca6682-0c28-473f-8122-fd19709cecfd None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:39.676546 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:39.732463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb2f6a-84cd-4c39-bcdd-85d1d3bda713 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:39.752664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca6682-0c28-473f-8122-fd19709cecfd None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:39.764356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:39.788927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e26a8b-5383-4a43-85af-c68be6ad38eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:39.789793 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e26a8b-5383-4a43-85af-c68be6ad38eb None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:39.790248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e26a8b-5383-4a43-85af-c68be6ad38eb None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:39.790721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e26a8b-5383-4a43-85af-c68be6ad38eb None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:39.791515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e26a8b-5383-4a43-85af-c68be6ad38eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:39.791991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e26a8b-5383-4a43-85af-c68be6ad38eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:39.793552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e26a8b-5383-4a43-85af-c68be6ad38eb None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:39.806235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 480/952] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:39.815438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e26a8b-5383-4a43-85af-c68be6ad38eb None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:39.837651 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb2f6a-84cd-4c39-bcdd-85d1d3bda713 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:39.874123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b860d-6efd-47b4-bf84-0714c0e34151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:39.874759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b860d-6efd-47b4-bf84-0714c0e34151 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:39.875039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b860d-6efd-47b4-bf84-0714c0e34151 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:39.875310 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b860d-6efd-47b4-bf84-0714c0e34151 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:39.875893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b860d-6efd-47b4-bf84-0714c0e34151 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:39.877305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b860d-6efd-47b4-bf84-0714c0e34151 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-f30b860d-6efd-47b4-bf84-0714c0e34151 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:39.886158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:39.887188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:39.888022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 473/953] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:39.893390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b860d-6efd-47b4-bf84-0714c0e34151 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:39.929505 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:39.943129 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e26a8b-5383-4a43-85af-c68be6ad38eb None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:39.983737 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131085d9-94ff-41af-a1ac-2f0395f1a877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:39.984444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131085d9-94ff-41af-a1ac-2f0395f1a877 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:39.984722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131085d9-94ff-41af-a1ac-2f0395f1a877 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:39.984990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131085d9-94ff-41af-a1ac-2f0395f1a877 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:39.992982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131085d9-94ff-41af-a1ac-2f0395f1a877 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:39.994432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131085d9-94ff-41af-a1ac-2f0395f1a877 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-131085d9-94ff-41af-a1ac-2f0395f1a877 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:40.007189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:40.008703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:40.009966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:40.009966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:40.009966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:40.009966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:40.009966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:40.009966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:40.009966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:40.009966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:40.009966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:06:40.009966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:40.010759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:40.012319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 481/954] 10.222.0.85 () {56 vars in 1117 bytes} [Mon Aug 29 20:06:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:40.014289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e168f09-a112-4518-9518-9dc6f9790283 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:40.014737 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e168f09-a112-4518-9518-9dc6f9790283 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:40.015012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e168f09-a112-4518-9518-9dc6f9790283 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:40.016805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 482/955] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:40.019523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e168f09-a112-4518-9518-9dc6f9790283 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:40.019645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b860d-6efd-47b4-bf84-0714c0e34151 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:40.056350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ecaba-1c88-43ab-94d8-8d427360081c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:40.056963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ecaba-1c88-43ab-94d8-8d427360081c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:40.057323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ecaba-1c88-43ab-94d8-8d427360081c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:40.057692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ecaba-1c88-43ab-94d8-8d427360081c None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:40.058509 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ecaba-1c88-43ab-94d8-8d427360081c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:40.058811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ecaba-1c88-43ab-94d8-8d427360081c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:40.060451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ecaba-1c88-43ab-94d8-8d427360081c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:40.067411 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 474/956] 10.222.0.85 () {56 vars in 1112 bytes} [Mon Aug 29 20:06:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:40.068811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa341103-6def-4134-8a8e-40a73f2b625b None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:40.069100 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa341103-6def-4134-8a8e-40a73f2b625b None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:40.069375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa341103-6def-4134-8a8e-40a73f2b625b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:40.092025 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-fa341103-6def-4134-8a8e-40a73f2b625b None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:40.093955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-fa341103-6def-4134-8a8e-40a73f2b625b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:40.135106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e168f09-a112-4518-9518-9dc6f9790283 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:40.170664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57896d-0c66-4ac5-bb1b-33b1bdb2ce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:40.171287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57896d-0c66-4ac5-bb1b-33b1bdb2ce1c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:40.171558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57896d-0c66-4ac5-bb1b-33b1bdb2ce1c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:40.171834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57896d-0c66-4ac5-bb1b-33b1bdb2ce1c None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:40.172599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57896d-0c66-4ac5-bb1b-33b1bdb2ce1c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:40.172876 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 475/957] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:40.173311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57896d-0c66-4ac5-bb1b-33b1bdb2ce1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:40.174332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57896d-0c66-4ac5-bb1b-33b1bdb2ce1c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:40.174436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa341103-6def-4134-8a8e-40a73f2b625b None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:40.203195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 483/958] 10.222.0.85 () {56 vars in 1149 bytes} [Mon Aug 29 20:06:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:40.205179 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57896d-0c66-4ac5-bb1b-33b1bdb2ce1c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:40.217618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:40.288778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:40.306535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa341103-6def-4134-8a8e-40a73f2b625b None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:40.322469 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57896d-0c66-4ac5-bb1b-33b1bdb2ce1c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:40.353854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3cfd8-fbea-4dbd-8eba-d8628881438c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:40.355071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3cfd8-fbea-4dbd-8eba-d8628881438c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:40.355947 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3cfd8-fbea-4dbd-8eba-d8628881438c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:40.356704 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3cfd8-fbea-4dbd-8eba-d8628881438c None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:40.358060 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3cfd8-fbea-4dbd-8eba-d8628881438c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:40.358565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3cfd8-fbea-4dbd-8eba-d8628881438c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:40.360130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3cfd8-fbea-4dbd-8eba-d8628881438c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:40.361022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91a8c0-4ca8-45f9-968e-11344f7912b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:40.361635 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91a8c0-4ca8-45f9-968e-11344f7912b9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:40.362053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91a8c0-4ca8-45f9-968e-11344f7912b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:40.362294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91a8c0-4ca8-45f9-968e-11344f7912b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:40.367393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 476/959] 10.222.0.85 () {56 vars in 1112 bytes} [Mon Aug 29 20:06:40 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:40.452263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91a8c0-4ca8-45f9-968e-11344f7912b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:40.454289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91a8c0-4ca8-45f9-968e-11344f7912b9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:40.467755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:40.469722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3cfd8-fbea-4dbd-8eba-d8628881438c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:40.580580 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 484/960] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:40 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:06:40.586701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91a8c0-4ca8-45f9-968e-11344f7912b9 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:40.600732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3cfd8-fbea-4dbd-8eba-d8628881438c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:40.638621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4e549-b6c1-452a-9cd3-4561312d2e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:40.639390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4e549-b6c1-452a-9cd3-4561312d2e03 None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:40.639669 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4e549-b6c1-452a-9cd3-4561312d2e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:40.639941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4e549-b6c1-452a-9cd3-4561312d2e03 None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:40.640626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4e549-b6c1-452a-9cd3-4561312d2e03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:40.642077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4e549-b6c1-452a-9cd3-4561312d2e03 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:40.644694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: WARNING keystone.server.flask.common [None req-a1e4e549-b6c1-452a-9cd3-4561312d2e03 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 20:06:40.688073 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.sql.core [None req-a1e4e549-b6c1-452a-9cd3-4561312d2e03 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 20:06:40.688073 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 20:06:40.688073 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [parameters: {'id': '6c6f32155829480eb8bb48d64226dfb7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 20:06:40.688073 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81858) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 20:06:40.695629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-a1e4e549-b6c1-452a-9cd3-4561312d2e03 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6c6f32155829480eb8bb48d64226dfb7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6c6f32155829480eb8bb48d64226dfb7. Aug 29 20:06:40.695629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:40.695629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:06:40.695629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:06:40.695629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:06:40.695629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:06:40.695629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:06:40.695629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:06:40.695629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:06:40.695629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:06:40.695629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:06:40.695629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:06:40.695629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application result.read() Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 29 20:06:40.696552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application yield resource Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise self.value Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application session.commit() Aug 29 20:06:40.697556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self.session.flush() Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application compat.raise_( Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 20:06:40.698489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 20:06:40.699430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application result.read() Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:06:40.700278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [parameters: {'id': '6c6f32155829480eb8bb48d64226dfb7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:40.701165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:40.703560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:40.704826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:40.704826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 29 20:06:40.704826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 29 20:06:40.704826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:40.704826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:40.704826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 29 20:06:40.704826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 29 20:06:40.704826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6c6f32155829480eb8bb48d64226dfb7. Aug 29 20:06:40.704826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:40.704826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 477/961] 10.222.0.85 () {60 vars in 1140 bytes} [Mon Aug 29 20:06:40 2022] POST /identity/v3/projects => generated 262 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:40.704826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d29dd9-d50a-4bb4-8c47-84573b9a4cc5 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:40.704826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d29dd9-d50a-4bb4-8c47-84573b9a4cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:40.705649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d29dd9-d50a-4bb4-8c47-84573b9a4cc5 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:40.705649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 478/962] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:40.705649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d29dd9-d50a-4bb4-8c47-84573b9a4cc5 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:40.715075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:40.721923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91a8c0-4ca8-45f9-968e-11344f7912b9 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:40.734682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:40.760183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc418ed-5012-43d4-9506-1a28b7d62aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:40.760815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc418ed-5012-43d4-9506-1a28b7d62aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:40.761095 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc418ed-5012-43d4-9506-1a28b7d62aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:40.761366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc418ed-5012-43d4-9506-1a28b7d62aa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:40.771286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc418ed-5012-43d4-9506-1a28b7d62aa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:40.772815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc418ed-5012-43d4-9506-1a28b7d62aa0 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-1dc418ed-5012-43d4-9506-1a28b7d62aa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:40.788462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:40.789736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:40.791002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:40.791002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:40.791002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:40.791002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 485/963] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:40.795430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bddf00-a827-4c0f-92d5-e5de34e5e15f None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:40.795716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bddf00-a827-4c0f-92d5-e5de34e5e15f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:40.795991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bddf00-a827-4c0f-92d5-e5de34e5e15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:40.815528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-28bddf00-a827-4c0f-92d5-e5de34e5e15f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:40.816903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-28bddf00-a827-4c0f-92d5-e5de34e5e15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:40.818951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d29dd9-d50a-4bb4-8c47-84573b9a4cc5 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:40.856949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243636fc-6161-41f7-94c9-d3af269744af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:40.857652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243636fc-6161-41f7-94c9-d3af269744af None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:40.858053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243636fc-6161-41f7-94c9-d3af269744af None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:40.858334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243636fc-6161-41f7-94c9-d3af269744af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:40.865414 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243636fc-6161-41f7-94c9-d3af269744af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:40.866988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243636fc-6161-41f7-94c9-d3af269744af None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-243636fc-6161-41f7-94c9-d3af269744af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:40.878609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:40.879711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:40.880622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:40.880622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:40.880622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:40.880622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:40.880622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:40.880622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:40.880622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:40.880622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:40.880622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 479/964] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:40.880622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243636fc-6161-41f7-94c9-d3af269744af None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:40.908210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 486/965] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:40.908210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bddf00-a827-4c0f-92d5-e5de34e5e15f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:40.998038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:40.999273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:41.007754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bddf00-a827-4c0f-92d5-e5de34e5e15f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:41.031418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243636fc-6161-41f7-94c9-d3af269744af None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:41.051804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ca10d-5726-43b9-9cf0-4c6de6611b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:41.052497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ca10d-5726-43b9-9cf0-4c6de6611b51 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:41.052775 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ca10d-5726-43b9-9cf0-4c6de6611b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:41.053046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ca10d-5726-43b9-9cf0-4c6de6611b51 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:41.053872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ca10d-5726-43b9-9cf0-4c6de6611b51 None admin] RBAC: Adding query filter params (name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:41.054224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ca10d-5726-43b9-9cf0-4c6de6611b51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:41.056067 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ca10d-5726-43b9-9cf0-4c6de6611b51 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:41.074699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509fb4d-0e23-4dbf-b92c-b5fc79691d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:41.075934 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509fb4d-0e23-4dbf-b92c-b5fc79691d44 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:41.076958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509fb4d-0e23-4dbf-b92c-b5fc79691d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:41.077534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509fb4d-0e23-4dbf-b92c-b5fc79691d44 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:41.078153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 487/966] 10.222.0.85 () {56 vars in 1112 bytes} [Mon Aug 29 20:06:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:41.078546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509fb4d-0e23-4dbf-b92c-b5fc79691d44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:41.078911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509fb4d-0e23-4dbf-b92c-b5fc79691d44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:41.084429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509fb4d-0e23-4dbf-b92c-b5fc79691d44 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:41.098250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 480/967] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:41.105954 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ca10d-5726-43b9-9cf0-4c6de6611b51 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:41.200946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509fb4d-0e23-4dbf-b92c-b5fc79691d44 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:41.226718 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ca10d-5726-43b9-9cf0-4c6de6611b51 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:41.278395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1e766-01f2-4726-8b2c-e5cabb500da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:41.280058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1e766-01f2-4726-8b2c-e5cabb500da0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:41.280788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1e766-01f2-4726-8b2c-e5cabb500da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:41.281518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1e766-01f2-4726-8b2c-e5cabb500da0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:41.293497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1e766-01f2-4726-8b2c-e5cabb500da0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:41.295616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1e766-01f2-4726-8b2c-e5cabb500da0 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-91d1e766-01f2-4726-8b2c-e5cabb500da0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:41.307000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:41.307929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:41.308908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:41.308908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:41.308908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:41.308908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:41.308908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:41.308908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:41.308908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:41.308908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:41.308908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 488/968] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:41.314561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1e766-01f2-4726-8b2c-e5cabb500da0 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:41.334310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:41.339808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509fb4d-0e23-4dbf-b92c-b5fc79691d44 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:41.376146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:41.388298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb243f-782f-41dc-ab62-6c4673ad72ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:41.389732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb243f-782f-41dc-ab62-6c4673ad72ad None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:41.390543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb243f-782f-41dc-ab62-6c4673ad72ad None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:41.391459 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb243f-782f-41dc-ab62-6c4673ad72ad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:41.392555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb243f-782f-41dc-ab62-6c4673ad72ad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:41.395104 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb243f-782f-41dc-ab62-6c4673ad72ad None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-6deb243f-782f-41dc-ab62-6c4673ad72ad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:41.410110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:41.411429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:41.412545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:41.415644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 481/969] 10.222.0.85 () {56 vars in 1126 bytes} [Mon Aug 29 20:06:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:41.423599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb243f-782f-41dc-ab62-6c4673ad72ad None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:41.447903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1e766-01f2-4726-8b2c-e5cabb500da0 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:41.485403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ef6f3-7199-4bdb-99ac-55ab42621a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:41.486453 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ef6f3-7199-4bdb-99ac-55ab42621a6a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:41.486968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ef6f3-7199-4bdb-99ac-55ab42621a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:41.487484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ef6f3-7199-4bdb-99ac-55ab42621a6a None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:41.488466 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ef6f3-7199-4bdb-99ac-55ab42621a6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:41.488996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ef6f3-7199-4bdb-99ac-55ab42621a6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:41.490687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ef6f3-7199-4bdb-99ac-55ab42621a6a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:41.505535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 489/970] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:41.512311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ef6f3-7199-4bdb-99ac-55ab42621a6a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:41.535019 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb243f-782f-41dc-ab62-6c4673ad72ad None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:41.580948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5160577-adc6-4f28-a773-096d46b1d210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:41.581673 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5160577-adc6-4f28-a773-096d46b1d210 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:41.582039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5160577-adc6-4f28-a773-096d46b1d210 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:41.582387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5160577-adc6-4f28-a773-096d46b1d210 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:41.583089 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5160577-adc6-4f28-a773-096d46b1d210 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:41.583461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5160577-adc6-4f28-a773-096d46b1d210 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:41.584935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5160577-adc6-4f28-a773-096d46b1d210 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:41.590760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 482/971] 10.222.0.85 () {56 vars in 1122 bytes} [Mon Aug 29 20:06:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:41.600349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5160577-adc6-4f28-a773-096d46b1d210 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:41.608036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:41.649407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ef6f3-7199-4bdb-99ac-55ab42621a6a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:41.672612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:41.687857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f504a9d-81eb-4bfc-b3dd-fa1117648a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:41.688788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f504a9d-81eb-4bfc-b3dd-fa1117648a57 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:41.689305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f504a9d-81eb-4bfc-b3dd-fa1117648a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:41.689801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f504a9d-81eb-4bfc-b3dd-fa1117648a57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:41.691097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f504a9d-81eb-4bfc-b3dd-fa1117648a57 None admin] RBAC: Adding query filter params (role.id=130698a8e0c6468491e7d81752709939, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=2d270713965b47fd9dbb2bcbec2634d5) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:41.691430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f504a9d-81eb-4bfc-b3dd-fa1117648a57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:41.692890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f504a9d-81eb-4bfc-b3dd-fa1117648a57 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:41.704523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 490/972] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:41 2022] GET /identity/v3/role_assignments?user.id=2d270713965b47fd9dbb2bcbec2634d5&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=130698a8e0c6468491e7d81752709939 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:41.726149 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5160577-adc6-4f28-a773-096d46b1d210 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:41.772048 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e65b34-a797-4ba4-b5af-9eb67c1ffb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:41.773405 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e65b34-a797-4ba4-b5af-9eb67c1ffb40 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:41.773923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e65b34-a797-4ba4-b5af-9eb67c1ffb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:41.774394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e65b34-a797-4ba4-b5af-9eb67c1ffb40 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:41.775551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e65b34-a797-4ba4-b5af-9eb67c1ffb40 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:41.776105 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e65b34-a797-4ba4-b5af-9eb67c1ffb40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:41.777823 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e65b34-a797-4ba4-b5af-9eb67c1ffb40 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:41.785251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 483/973] 10.222.0.85 () {56 vars in 1122 bytes} [Mon Aug 29 20:06:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:41.790670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f504a9d-81eb-4bfc-b3dd-fa1117648a57 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:41.902891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f504a9d-81eb-4bfc-b3dd-fa1117648a57 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:41.939674 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fa0b6-80fb-405b-b1b9-7798d724026b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:41.940817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fa0b6-80fb-405b-b1b9-7798d724026b None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:41.950124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fa0b6-80fb-405b-b1b9-7798d724026b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:41.950124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fa0b6-80fb-405b-b1b9-7798d724026b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:41.950124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9fa0b6-80fb-405b-b1b9-7798d724026b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:41.950124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9fa0b6-80fb-405b-b1b9-7798d724026b None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:41.950492 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:42.112664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 491/974] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:06:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:06:42.377877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee9f87-64f0-44ba-a49c-5161ece1d043 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:42.378827 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee9f87-64f0-44ba-a49c-5161ece1d043 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:42.379322 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee9f87-64f0-44ba-a49c-5161ece1d043 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:42.381154 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 484/975] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:42.386677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b57f22-63da-4d95-bf2e-9d6842a09d89 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:42.387344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b57f22-63da-4d95-bf2e-9d6842a09d89 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:42.387831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b57f22-63da-4d95-bf2e-9d6842a09d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:42.409960 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-b5b57f22-63da-4d95-bf2e-9d6842a09d89 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:42.411489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b5b57f22-63da-4d95-bf2e-9d6842a09d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:42.487339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 492/976] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:42.760906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee9f87-64f0-44ba-a49c-5161ece1d043 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:42.873801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee9f87-64f0-44ba-a49c-5161ece1d043 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:42.894121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:42.910391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5a3ec-fb6b-41e4-bdae-f05a26a4bb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:42.911260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5a3ec-fb6b-41e4-bdae-f05a26a4bb97 None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:42.911692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5a3ec-fb6b-41e4-bdae-f05a26a4bb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:42.912115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5a3ec-fb6b-41e4-bdae-f05a26a4bb97 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:42.912943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5a3ec-fb6b-41e4-bdae-f05a26a4bb97 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:42.914464 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5a3ec-fb6b-41e4-bdae-f05a26a4bb97 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:43.021518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 485/977] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:06:42 2022] POST /identity/v3/services => generated 242 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:06:43.171527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc93d5-9823-40cf-891f-6f448b2672f7 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:43.172131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc93d5-9823-40cf-891f-6f448b2672f7 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:43.172131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc93d5-9823-40cf-891f-6f448b2672f7 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:43.173705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 493/978] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:43.179105 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2922b5-ce97-4129-a21b-2b4ab1748257 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:43.179400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2922b5-ce97-4129-a21b-2b4ab1748257 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:43.179679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2922b5-ce97-4129-a21b-2b4ab1748257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:43.200393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-bb2922b5-ce97-4129-a21b-2b4ab1748257 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:43.202583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bb2922b5-ce97-4129-a21b-2b4ab1748257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:43.292121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 486/979] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:43.565605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc93d5-9823-40cf-891f-6f448b2672f7 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:43.677505 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc93d5-9823-40cf-891f-6f448b2672f7 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:43.696586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:43.713770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c106b8-c6b9-4051-86bc-8b8a72406a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:43.715034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c106b8-c6b9-4051-86bc-8b8a72406a34 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:43.715034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c106b8-c6b9-4051-86bc-8b8a72406a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:43.715034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c106b8-c6b9-4051-86bc-8b8a72406a34 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:43.715639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c106b8-c6b9-4051-86bc-8b8a72406a34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:43.717740 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c106b8-c6b9-4051-86bc-8b8a72406a34 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:43.720324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: WARNING keystone.server.flask.common [None req-45c106b8-c6b9-4051-86bc-8b8a72406a34 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 20:06:43.790599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.sql.core [None req-45c106b8-c6b9-4051-86bc-8b8a72406a34 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 20:06:43.790599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 20:06:43.790599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [parameters: {'user_id': '67094511027b43a5a108276b0e4b9563', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 20:06:43.790599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81859) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-45c106b8-c6b9-4051-86bc-8b8a72406a34 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application result.read() Aug 29 20:06:43.808879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:06:43.809916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application yield resource Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise self.value Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application session.commit() Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 20:06:43.811995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self.session.flush() Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application compat.raise_( Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 20:06:43.813109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:06:43.814000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application result.read() Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:06:43.814934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [parameters: {'user_id': '67094511027b43a5a108276b0e4b9563', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:43.816680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 29 20:06:43.817699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 29 20:06:43.818795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 29 20:06:43.818795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 29 20:06:43.818795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 20:06:43.818795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 20:06:43.818795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 20:06:43.818795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:43.818795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 494/980] 10.222.0.85 () {60 vars in 1131 bytes} [Mon Aug 29 20:06:43 2022] POST /identity/v3/users => generated 160 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:43.823050 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1f71f-f371-4043-ad3b-67fdd4d868e6 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:43.823050 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1f71f-f371-4043-ad3b-67fdd4d868e6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:43.823050 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1f71f-f371-4043-ad3b-67fdd4d868e6 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:43.823050 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 487/981] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:43.823694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c106b8-c6b9-4051-86bc-8b8a72406a34 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:43.823779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f78404-30be-4f51-8e9c-5666a452abae None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:43.824067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f78404-30be-4f51-8e9c-5666a452abae None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:43.824397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f78404-30be-4f51-8e9c-5666a452abae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:43.848563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e1f78404-30be-4f51-8e9c-5666a452abae None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:43.849765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e1f78404-30be-4f51-8e9c-5666a452abae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:43.931950 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 488/982] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:43.952131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c106b8-c6b9-4051-86bc-8b8a72406a34 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:43.988283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1964b-8709-4f4f-8b8a-a800ac813d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:43.988968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1964b-8709-4f4f-8b8a-a800ac813d12 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:43.989293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1964b-8709-4f4f-8b8a-a800ac813d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:43.989587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1964b-8709-4f4f-8b8a-a800ac813d12 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:43.997157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1964b-8709-4f4f-8b8a-a800ac813d12 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:43.999553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1964b-8709-4f4f-8b8a-a800ac813d12 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-8cc1964b-8709-4f4f-8b8a-a800ac813d12 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:44.012090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:44.013138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:44.014612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:44.014612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:44.014612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:44.014612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:44.014612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:44.014612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:44.014612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:44.014612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:44.014612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:06:44.014612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:44.017168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 495/983] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:06:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:44.029363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f78404-30be-4f51-8e9c-5666a452abae None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:44.066117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:44.067820 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901035da-2a3e-4665-840d-4157d1ffd7aa None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:44.068057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901035da-2a3e-4665-840d-4157d1ffd7aa None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:44.068384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901035da-2a3e-4665-840d-4157d1ffd7aa None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:44.076696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 496/984] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:44.078624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa47291-355b-4497-9be3-f7779c4572fe None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:44.078916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa47291-355b-4497-9be3-f7779c4572fe None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:44.079189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa47291-355b-4497-9be3-f7779c4572fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:44.098910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-6fa47291-355b-4497-9be3-f7779c4572fe None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:44.100501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-6fa47291-355b-4497-9be3-f7779c4572fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:44.137352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f78404-30be-4f51-8e9c-5666a452abae None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:44.138482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:44.176820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e19c7f-d805-4318-b043-c09ebb6fbc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:44.177433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e19c7f-d805-4318-b043-c09ebb6fbc22 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:44.177710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e19c7f-d805-4318-b043-c09ebb6fbc22 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:44.177983 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e19c7f-d805-4318-b043-c09ebb6fbc22 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:44.178682 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e19c7f-d805-4318-b043-c09ebb6fbc22 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:44.179045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e19c7f-d805-4318-b043-c09ebb6fbc22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:44.180571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e19c7f-d805-4318-b043-c09ebb6fbc22 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:44.183819 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 497/985] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:44.219287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 489/986] 10.222.0.85 () {56 vars in 1101 bytes} [Mon Aug 29 20:06:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:44.222388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa47291-355b-4497-9be3-f7779c4572fe None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:44.337582 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa47291-355b-4497-9be3-f7779c4572fe None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:44.379075 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc26987-0cc7-4481-a7cd-b2edb6f58d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:44.379711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26987-0cc7-4481-a7cd-b2edb6f58d6b None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:44.380006 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26987-0cc7-4481-a7cd-b2edb6f58d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:44.380301 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26987-0cc7-4481-a7cd-b2edb6f58d6b None admin] PATH_INFO: `/v3/services/image` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:44.380877 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc26987-0cc7-4481-a7cd-b2edb6f58d6b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:44.382248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc26987-0cc7-4481-a7cd-b2edb6f58d6b None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-2cc26987-0cc7-4481-a7cd-b2edb6f58d6b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:44.395839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:44.396976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:06:44.397811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:44.398616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 498/987] 10.222.0.85 () {56 vars in 1105 bytes} [Mon Aug 29 20:06:44 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:44.405991 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e19c7f-d805-4318-b043-c09ebb6fbc22 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:44.471084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc26987-0cc7-4481-a7cd-b2edb6f58d6b None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:44.520232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:44.522538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:44.528632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e19c7f-d805-4318-b043-c09ebb6fbc22 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:44.569582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221d13d-2bec-4ace-8eea-dca9be107539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:44.570185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221d13d-2bec-4ace-8eea-dca9be107539 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:44.570512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221d13d-2bec-4ace-8eea-dca9be107539 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:44.570722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221d13d-2bec-4ace-8eea-dca9be107539 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:44.571611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221d13d-2bec-4ace-8eea-dca9be107539 None admin] RBAC: Adding query filter params (name=image) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:44.571818 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221d13d-2bec-4ace-8eea-dca9be107539 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:44.573214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221d13d-2bec-4ace-8eea-dca9be107539 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:44.578447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 490/988] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:44.584577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221d13d-2bec-4ace-8eea-dca9be107539 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:44.592530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc26987-0cc7-4481-a7cd-b2edb6f58d6b None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:44.631941 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23084da3-463e-46e6-90c3-fcce2a617097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:44.632863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23084da3-463e-46e6-90c3-fcce2a617097 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:44.633383 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23084da3-463e-46e6-90c3-fcce2a617097 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:44.633796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23084da3-463e-46e6-90c3-fcce2a617097 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:44.634522 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23084da3-463e-46e6-90c3-fcce2a617097 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:44.635969 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23084da3-463e-46e6-90c3-fcce2a617097 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-23084da3-463e-46e6-90c3-fcce2a617097 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:44.645963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:44.646982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:44.647848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:44.652343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 499/989] 10.222.0.85 () {56 vars in 1126 bytes} [Mon Aug 29 20:06:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:44.653158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23084da3-463e-46e6-90c3-fcce2a617097 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:44.702679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221d13d-2bec-4ace-8eea-dca9be107539 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:44.739924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a2a64-7a63-4f79-a1ca-a13c7c6ef368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:44.740566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a2a64-7a63-4f79-a1ca-a13c7c6ef368 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:44.740844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a2a64-7a63-4f79-a1ca-a13c7c6ef368 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:44.741114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a2a64-7a63-4f79-a1ca-a13c7c6ef368 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:44.741747 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a2a64-7a63-4f79-a1ca-a13c7c6ef368 None admin] RBAC: Adding query filter params (type=image) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:44.742047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a2a64-7a63-4f79-a1ca-a13c7c6ef368 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:44.743411 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a2a64-7a63-4f79-a1ca-a13c7c6ef368 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:44.750402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 491/990] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:44 2022] GET /identity/v3/services?type=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:44.768232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:44.772541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23084da3-463e-46e6-90c3-fcce2a617097 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:44.810341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd282f4-86e4-4a45-9c90-d53ab89ffca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:44.811989 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd282f4-86e4-4a45-9c90-d53ab89ffca3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:44.812318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd282f4-86e4-4a45-9c90-d53ab89ffca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:44.812587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd282f4-86e4-4a45-9c90-d53ab89ffca3 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:44.813249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd282f4-86e4-4a45-9c90-d53ab89ffca3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:44.813555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd282f4-86e4-4a45-9c90-d53ab89ffca3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:44.814960 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd282f4-86e4-4a45-9c90-d53ab89ffca3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:44.824240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 500/991] 10.222.0.85 () {56 vars in 1122 bytes} [Mon Aug 29 20:06:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:44.827249 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a2a64-7a63-4f79-a1ca-a13c7c6ef368 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:44.854828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:44.936880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a2a64-7a63-4f79-a1ca-a13c7c6ef368 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:44.977408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5378bbb-bedf-423b-b550-c8b9e4064907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:44.978024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5378bbb-bedf-423b-b550-c8b9e4064907 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:44.978301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5378bbb-bedf-423b-b550-c8b9e4064907 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:44.978569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5378bbb-bedf-423b-b550-c8b9e4064907 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:44.979198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5378bbb-bedf-423b-b550-c8b9e4064907 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:44.979499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5378bbb-bedf-423b-b550-c8b9e4064907 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:44.981012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5378bbb-bedf-423b-b550-c8b9e4064907 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:44.986954 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 492/992] 10.222.0.85 () {56 vars in 1122 bytes} [Mon Aug 29 20:06:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:44.992867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd282f4-86e4-4a45-9c90-d53ab89ffca3 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:45.096555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd282f4-86e4-4a45-9c90-d53ab89ffca3 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:45.100823 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2311f-4780-417b-af39-d0078d60a591 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:45.101110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2311f-4780-417b-af39-d0078d60a591 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:45.101386 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2311f-4780-417b-af39-d0078d60a591 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:45.102875 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 493/993] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:45.108659 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09021741-211a-4142-ab08-15418ec78e65 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:45.108861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09021741-211a-4142-ab08-15418ec78e65 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:45.109128 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09021741-211a-4142-ab08-15418ec78e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:45.128109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-09021741-211a-4142-ab08-15418ec78e65 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:45.129637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-09021741-211a-4142-ab08-15418ec78e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:45.135809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e00ee-4435-4463-9efe-cdbaec024b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:45.136448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e00ee-4435-4463-9efe-cdbaec024b36 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:45.136727 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e00ee-4435-4463-9efe-cdbaec024b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:45.137003 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e00ee-4435-4463-9efe-cdbaec024b36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:45.137709 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e00ee-4435-4463-9efe-cdbaec024b36 None admin] RBAC: Adding query filter params (interface=admin, service_id=c732d9974c3a4e80bc12b9315cb2653a, region_id=RegionOne) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:45.138016 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e00ee-4435-4463-9efe-cdbaec024b36 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:45.139934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e00ee-4435-4463-9efe-cdbaec024b36 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:45.145299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 501/994] 10.222.0.85 () {56 vars in 1249 bytes} [Mon Aug 29 20:06:44 2022] GET /identity/v3/endpoints?service_id=c732d9974c3a4e80bc12b9315cb2653a&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:45.150910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e00ee-4435-4463-9efe-cdbaec024b36 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:45.179920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:45.206615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 494/995] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:45.259431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e00ee-4435-4463-9efe-cdbaec024b36 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:45.293090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f53eef-4dff-47c3-9dbb-ebaad5779678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:45.293626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f53eef-4dff-47c3-9dbb-ebaad5779678 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:45.293907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f53eef-4dff-47c3-9dbb-ebaad5779678 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:45.294172 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f53eef-4dff-47c3-9dbb-ebaad5779678 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:45.294784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f53eef-4dff-47c3-9dbb-ebaad5779678 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:45.296265 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f53eef-4dff-47c3-9dbb-ebaad5779678 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:45.310381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 502/996] 10.222.0.85 () {56 vars in 1087 bytes} [Mon Aug 29 20:06:45 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:06:45.478244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09021741-211a-4142-ab08-15418ec78e65 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:45.495911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:45.578218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09021741-211a-4142-ab08-15418ec78e65 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:45.616564 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1df32d-b1da-45bd-af73-e24a4e69a8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:45.617207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1df32d-b1da-45bd-af73-e24a4e69a8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:45.617485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1df32d-b1da-45bd-af73-e24a4e69a8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:45.617752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1df32d-b1da-45bd-af73-e24a4e69a8f3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:45.618332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1df32d-b1da-45bd-af73-e24a4e69a8f3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:45.619697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1df32d-b1da-45bd-af73-e24a4e69a8f3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-7e1df32d-b1da-45bd-af73-e24a4e69a8f3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:45.628838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:45.629858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:45.630787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 495/997] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:45.635695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f53eef-4dff-47c3-9dbb-ebaad5779678 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:45.729955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:45.740640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f53eef-4dff-47c3-9dbb-ebaad5779678 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:45.776405 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e40e84-7c7d-4c63-85e5-13a4204b562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:45.777015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e40e84-7c7d-4c63-85e5-13a4204b562f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:45.777292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e40e84-7c7d-4c63-85e5-13a4204b562f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:45.777560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e40e84-7c7d-4c63-85e5-13a4204b562f None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:45.780414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e40e84-7c7d-4c63-85e5-13a4204b562f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:45.780414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e40e84-7c7d-4c63-85e5-13a4204b562f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:45.780414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e40e84-7c7d-4c63-85e5-13a4204b562f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:45.785729 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 503/998] 10.222.0.85 () {56 vars in 1112 bytes} [Mon Aug 29 20:06:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:45.791230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1df32d-b1da-45bd-af73-e24a4e69a8f3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:45.876109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:45.895595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1df32d-b1da-45bd-af73-e24a4e69a8f3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:45.929635 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f6364-4c21-4493-b8d3-4abdb5f61a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:45.930238 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f6364-4c21-4493-b8d3-4abdb5f61a31 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:45.930517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f6364-4c21-4493-b8d3-4abdb5f61a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:45.930787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f6364-4c21-4493-b8d3-4abdb5f61a31 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:45.931417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f6364-4c21-4493-b8d3-4abdb5f61a31 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:45.931718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f6364-4c21-4493-b8d3-4abdb5f61a31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:45.933171 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f6364-4c21-4493-b8d3-4abdb5f61a31 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:45.939176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 496/999] 10.222.0.85 () {56 vars in 1112 bytes} [Mon Aug 29 20:06:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:45.944896 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e40e84-7c7d-4c63-85e5-13a4204b562f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:46.050186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e40e84-7c7d-4c63-85e5-13a4204b562f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:46.083227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a0943-80b8-48ec-860a-54c6ffeee808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:46.083830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a0943-80b8-48ec-860a-54c6ffeee808 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:46.084104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a0943-80b8-48ec-860a-54c6ffeee808 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:46.084401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a0943-80b8-48ec-860a-54c6ffeee808 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:46.085096 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a0943-80b8-48ec-860a-54c6ffeee808 None admin] RBAC: Adding query filter params (interface=public, service_id=4158184a1c784e1f931d8372f5b4da61, region_id=RegionOne) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:46.085421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a0943-80b8-48ec-860a-54c6ffeee808 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:46.086770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a0943-80b8-48ec-860a-54c6ffeee808 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:46.094796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 504/1000] 10.222.0.85 () {56 vars in 1251 bytes} [Mon Aug 29 20:06:45 2022] GET /identity/v3/endpoints?service_id=4158184a1c784e1f931d8372f5b4da61&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:46.100105 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f6364-4c21-4493-b8d3-4abdb5f61a31 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:46.205650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f6364-4c21-4493-b8d3-4abdb5f61a31 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:46.240712 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f028f39-bf5e-485b-9ed6-7850bbc66e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:46.241314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f028f39-bf5e-485b-9ed6-7850bbc66e60 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:46.241590 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f028f39-bf5e-485b-9ed6-7850bbc66e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:46.241859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f028f39-bf5e-485b-9ed6-7850bbc66e60 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:46.242456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f028f39-bf5e-485b-9ed6-7850bbc66e60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:46.243808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f028f39-bf5e-485b-9ed6-7850bbc66e60 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:46.256209 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 497/1001] 10.222.0.85 () {56 vars in 1087 bytes} [Mon Aug 29 20:06:46 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:06:46.690845 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fbfaf-d95a-4f1b-821d-0c4afb49d258 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:46.691395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fbfaf-d95a-4f1b-821d-0c4afb49d258 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:46.691395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fbfaf-d95a-4f1b-821d-0c4afb49d258 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:46.692906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 505/1002] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:46.697947 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702eed89-f541-4276-a8ff-0638063b3a9b None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:46.698238 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702eed89-f541-4276-a8ff-0638063b3a9b None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:46.698515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702eed89-f541-4276-a8ff-0638063b3a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:46.717812 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-702eed89-f541-4276-a8ff-0638063b3a9b None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:46.719002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-702eed89-f541-4276-a8ff-0638063b3a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:46.721231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:46.789229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 498/1003] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:47.062427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fbfaf-d95a-4f1b-821d-0c4afb49d258 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:47.083273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:47.169189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fbfaf-d95a-4f1b-821d-0c4afb49d258 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:47.204409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d843029-e7cc-4c02-a5a1-93e91c89a3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:47.205020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d843029-e7cc-4c02-a5a1-93e91c89a3cb None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:47.205298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d843029-e7cc-4c02-a5a1-93e91c89a3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:47.205568 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d843029-e7cc-4c02-a5a1-93e91c89a3cb None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:47.206227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d843029-e7cc-4c02-a5a1-93e91c89a3cb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:47.207592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d843029-e7cc-4c02-a5a1-93e91c89a3cb None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:47.279342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d76cd-a30b-4c33-8849-2e8850e6522c None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:47.279874 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d76cd-a30b-4c33-8849-2e8850e6522c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:47.279874 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d76cd-a30b-4c33-8849-2e8850e6522c None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:47.281290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 499/1004] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:47.286397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0de5d-fbba-424e-aa7b-d5278bc85889 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:47.286683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0de5d-fbba-424e-aa7b-d5278bc85889 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:47.286952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0de5d-fbba-424e-aa7b-d5278bc85889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:47.304871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3ef0de5d-fbba-424e-aa7b-d5278bc85889 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:47.306065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3ef0de5d-fbba-424e-aa7b-d5278bc85889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:47.334338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 506/1005] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:06:47 2022] POST /identity/v3/services => generated 242 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:06:47.397301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 500/1006] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:47 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:47.669095 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d843029-e7cc-4c02-a5a1-93e91c89a3cb None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:47.752376 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:47.774021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d843029-e7cc-4c02-a5a1-93e91c89a3cb None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:47.809021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2fbc5-0963-4637-af57-b0a0d7658af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:47.809652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2fbc5-0963-4637-af57-b0a0d7658af5 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:47.809927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2fbc5-0963-4637-af57-b0a0d7658af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:47.810203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2fbc5-0963-4637-af57-b0a0d7658af5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:47.811401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2fbc5-0963-4637-af57-b0a0d7658af5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:47.813082 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2fbc5-0963-4637-af57-b0a0d7658af5 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-eab2fbc5-0963-4637-af57-b0a0d7658af5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:47.822217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:47.823227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:06:47.824084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:47.828350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 507/1007] 10.222.0.85 () {56 vars in 1126 bytes} [Mon Aug 29 20:06:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:47.829612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0de5d-fbba-424e-aa7b-d5278bc85889 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:47.894314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:47.931485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0de5d-fbba-424e-aa7b-d5278bc85889 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:47.967024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115b928-38cb-4ee4-9f9b-98807a172926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:47.967665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115b928-38cb-4ee4-9f9b-98807a172926 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:47.967943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115b928-38cb-4ee4-9f9b-98807a172926 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:47.968234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115b928-38cb-4ee4-9f9b-98807a172926 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:47.968860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115b928-38cb-4ee4-9f9b-98807a172926 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:47.969160 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115b928-38cb-4ee4-9f9b-98807a172926 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:47.970518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115b928-38cb-4ee4-9f9b-98807a172926 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:47.980579 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 501/1008] 10.222.0.85 () {56 vars in 1122 bytes} [Mon Aug 29 20:06:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:47.981840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2fbc5-0963-4637-af57-b0a0d7658af5 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:48.143640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2fbc5-0963-4637-af57-b0a0d7658af5 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:48.172573 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fa136-343a-4063-ae00-825cc213b3b9 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:48.172882 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fa136-343a-4063-ae00-825cc213b3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:48.173121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fa136-343a-4063-ae00-825cc213b3b9 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:48.174598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 502/1009] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:48.178113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76039992-ee69-4b0e-b75e-92c6faec6da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:48.178722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76039992-ee69-4b0e-b75e-92c6faec6da0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:48.179002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76039992-ee69-4b0e-b75e-92c6faec6da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:48.179117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da727c-f1f2-4587-b7e6-48c33f74905d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:48.179269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76039992-ee69-4b0e-b75e-92c6faec6da0 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:48.179372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da727c-f1f2-4587-b7e6-48c33f74905d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:48.179648 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da727c-f1f2-4587-b7e6-48c33f74905d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:48.179894 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76039992-ee69-4b0e-b75e-92c6faec6da0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:48.180226 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76039992-ee69-4b0e-b75e-92c6faec6da0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:48.181578 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76039992-ee69-4b0e-b75e-92c6faec6da0 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:48.188575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 508/1010] 10.222.0.85 () {56 vars in 1122 bytes} [Mon Aug 29 20:06:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:48.194059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76039992-ee69-4b0e-b75e-92c6faec6da0 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:48.199115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-08da727c-f1f2-4587-b7e6-48c33f74905d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:48.200319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-08da727c-f1f2-4587-b7e6-48c33f74905d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:48.204690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:48.274514 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 503/1011] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:48 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:48.299501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76039992-ee69-4b0e-b75e-92c6faec6da0 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:48.336266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b918a48-3a52-43a3-8dec-0b8b62168594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:48.336986 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b918a48-3a52-43a3-8dec-0b8b62168594 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:48.337339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b918a48-3a52-43a3-8dec-0b8b62168594 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:48.337656 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b918a48-3a52-43a3-8dec-0b8b62168594 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:48.338453 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b918a48-3a52-43a3-8dec-0b8b62168594 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:48.340055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b918a48-3a52-43a3-8dec-0b8b62168594 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:48.484282 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 509/1012] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:06:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:06:48.544128 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da727c-f1f2-4587-b7e6-48c33f74905d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:48.615397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:48.657838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da727c-f1f2-4587-b7e6-48c33f74905d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:48.699909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173bc73-617a-47ef-9b0b-b0d5fe969add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:48.700580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173bc73-617a-47ef-9b0b-b0d5fe969add None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:48.701377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173bc73-617a-47ef-9b0b-b0d5fe969add None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:48.701728 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173bc73-617a-47ef-9b0b-b0d5fe969add None admin] PATH_INFO: `/v3/services/compute` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:48.702309 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173bc73-617a-47ef-9b0b-b0d5fe969add None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:48.703671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173bc73-617a-47ef-9b0b-b0d5fe969add None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-c173bc73-617a-47ef-9b0b-b0d5fe969add None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:48.712799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:48.713790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:48.714701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 504/1013] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:48.722309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b918a48-3a52-43a3-8dec-0b8b62168594 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:48.791330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:48.827960 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b918a48-3a52-43a3-8dec-0b8b62168594 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:48.864924 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6520f78-4e8e-4bb5-aa9c-6248367a56d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:48.865541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6520f78-4e8e-4bb5-aa9c-6248367a56d9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:48.865822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6520f78-4e8e-4bb5-aa9c-6248367a56d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:48.866099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6520f78-4e8e-4bb5-aa9c-6248367a56d9 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:48.866736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6520f78-4e8e-4bb5-aa9c-6248367a56d9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:48.867066 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6520f78-4e8e-4bb5-aa9c-6248367a56d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:48.868749 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6520f78-4e8e-4bb5-aa9c-6248367a56d9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:48.875521 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 510/1014] 10.222.0.85 () {56 vars in 1112 bytes} [Mon Aug 29 20:06:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:48.880991 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173bc73-617a-47ef-9b0b-b0d5fe969add None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:48.986230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173bc73-617a-47ef-9b0b-b0d5fe969add None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:49.004686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:49.022316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c11de-3094-4d90-81c3-116cae2422c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:49.022980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c11de-3094-4d90-81c3-116cae2422c4 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:49.023261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c11de-3094-4d90-81c3-116cae2422c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:49.023647 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c11de-3094-4d90-81c3-116cae2422c4 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:49.024313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c11de-3094-4d90-81c3-116cae2422c4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:49.024621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c11de-3094-4d90-81c3-116cae2422c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:49.026256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c11de-3094-4d90-81c3-116cae2422c4 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:49.036430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 505/1015] 10.222.0.85 () {56 vars in 1112 bytes} [Mon Aug 29 20:06:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:49.038220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6520f78-4e8e-4bb5-aa9c-6248367a56d9 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:49.144546 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6520f78-4e8e-4bb5-aa9c-6248367a56d9 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:49.175281 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:49.181585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c05ba3-46e5-4578-98db-6b70b90eb18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:49.182196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c05ba3-46e5-4578-98db-6b70b90eb18b None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:49.182488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c05ba3-46e5-4578-98db-6b70b90eb18b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:49.182765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c05ba3-46e5-4578-98db-6b70b90eb18b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:49.183439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c05ba3-46e5-4578-98db-6b70b90eb18b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:49.185131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c05ba3-46e5-4578-98db-6b70b90eb18b None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:49.270494 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f02f2e-01a7-4aaf-878f-4f93f1faa5f3 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:49.270778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f02f2e-01a7-4aaf-878f-4f93f1faa5f3 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:49.271117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f02f2e-01a7-4aaf-878f-4f93f1faa5f3 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:49.272547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 506/1016] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:49.277665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edea66c-cb5b-44ed-838e-283f1108ffd9 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:49.277952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edea66c-cb5b-44ed-838e-283f1108ffd9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:49.278232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edea66c-cb5b-44ed-838e-283f1108ffd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:49.296971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-7edea66c-cb5b-44ed-838e-283f1108ffd9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:49.298166 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7edea66c-cb5b-44ed-838e-283f1108ffd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:49.339930 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 511/1017] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:06:49 2022] POST /identity/v3/endpoints => generated 334 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:06:49.384902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 507/1018] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:49.657786 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c05ba3-46e5-4578-98db-6b70b90eb18b None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:49.763730 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c05ba3-46e5-4578-98db-6b70b90eb18b None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:49.798048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b34fd-eb7c-4d69-b9ed-9c185613465e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:49.798685 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b34fd-eb7c-4d69-b9ed-9c185613465e None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:49.798943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b34fd-eb7c-4d69-b9ed-9c185613465e None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:49.799215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b34fd-eb7c-4d69-b9ed-9c185613465e None admin] PATH_INFO: `/v3/services/image` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:49.799791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b34fd-eb7c-4d69-b9ed-9c185613465e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:49.801178 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b34fd-eb7c-4d69-b9ed-9c185613465e None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-aa1b34fd-eb7c-4d69-b9ed-9c185613465e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:49.810065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:49.811531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:06:49.812408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:49.814224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 512/1019] 10.222.0.85 () {56 vars in 1105 bytes} [Mon Aug 29 20:06:49 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:49.817932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edea66c-cb5b-44ed-838e-283f1108ffd9 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:49.923722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edea66c-cb5b-44ed-838e-283f1108ffd9 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:49.933948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:49.960342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0460f73-d524-4dc6-ab28-fbf3fb01fa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:49.960976 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0460f73-d524-4dc6-ab28-fbf3fb01fa1a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:49.961261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0460f73-d524-4dc6-ab28-fbf3fb01fa1a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:49.961537 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0460f73-d524-4dc6-ab28-fbf3fb01fa1a None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:49.962174 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0460f73-d524-4dc6-ab28-fbf3fb01fa1a None admin] RBAC: Adding query filter params (name=image) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:49.962910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0460f73-d524-4dc6-ab28-fbf3fb01fa1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:49.964753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0460f73-d524-4dc6-ab28-fbf3fb01fa1a None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:49.975123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 508/1020] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:49.976271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b34fd-eb7c-4d69-b9ed-9c185613465e None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:50.028295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:50.081009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b34fd-eb7c-4d69-b9ed-9c185613465e None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:50.116188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ca268-82a4-4531-a2fe-c2c23d2f7f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:50.116821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ca268-82a4-4531-a2fe-c2c23d2f7f7c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:50.117100 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ca268-82a4-4531-a2fe-c2c23d2f7f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:50.117371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ca268-82a4-4531-a2fe-c2c23d2f7f7c None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:50.117999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ca268-82a4-4531-a2fe-c2c23d2f7f7c None admin] RBAC: Adding query filter params (type=image) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:50.118301 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ca268-82a4-4531-a2fe-c2c23d2f7f7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:50.119660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ca268-82a4-4531-a2fe-c2c23d2f7f7c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:50.128258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 513/1021] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:50.130988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0460f73-d524-4dc6-ab28-fbf3fb01fa1a None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:50.233684 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0460f73-d524-4dc6-ab28-fbf3fb01fa1a None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:50.270610 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f109e05-a43c-4ba2-9229-dfcd05d4c3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:50.271216 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f109e05-a43c-4ba2-9229-dfcd05d4c3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:50.271496 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f109e05-a43c-4ba2-9229-dfcd05d4c3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:50.271769 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f109e05-a43c-4ba2-9229-dfcd05d4c3c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:50.272493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f109e05-a43c-4ba2-9229-dfcd05d4c3c9 None admin] RBAC: Adding query filter params (interface=public, service_id=f3f90dbc3a554f7e8607707799b73b8d, region_id=RegionOne) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:50.272801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f109e05-a43c-4ba2-9229-dfcd05d4c3c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:50.274230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f109e05-a43c-4ba2-9229-dfcd05d4c3c9 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:50.285000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 509/1022] 10.222.0.85 () {56 vars in 1251 bytes} [Mon Aug 29 20:06:50 2022] GET /identity/v3/endpoints?service_id=f3f90dbc3a554f7e8607707799b73b8d&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:50.285809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ca268-82a4-4531-a2fe-c2c23d2f7f7c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:50.388204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ca268-82a4-4531-a2fe-c2c23d2f7f7c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:50.402614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:50.427454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a0e44-0538-47eb-a989-5173831c7a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:50.428063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a0e44-0538-47eb-a989-5173831c7a00 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:50.428373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a0e44-0538-47eb-a989-5173831c7a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:50.428647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a0e44-0538-47eb-a989-5173831c7a00 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:50.429252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a0e44-0538-47eb-a989-5173831c7a00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:50.430608 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a0e44-0538-47eb-a989-5173831c7a00 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:50.437484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 514/1023] 10.222.0.85 () {56 vars in 1087 bytes} [Mon Aug 29 20:06:50 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:06:50.456594 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892d28-b4eb-49dc-8cb7-238294536368 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:50.456883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892d28-b4eb-49dc-8cb7-238294536368 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:50.457207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892d28-b4eb-49dc-8cb7-238294536368 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:50.459166 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 510/1024] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:50.465404 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414125ef-6db4-4a81-ab90-da3298946580 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:50.465673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414125ef-6db4-4a81-ab90-da3298946580 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:50.465995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414125ef-6db4-4a81-ab90-da3298946580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:50.487576 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-414125ef-6db4-4a81-ab90-da3298946580 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:50.489842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-414125ef-6db4-4a81-ab90-da3298946580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:50.567480 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 515/1025] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:50.836202 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08892d28-b4eb-49dc-8cb7-238294536368 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:50.878523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:50.947490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08892d28-b4eb-49dc-8cb7-238294536368 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:50.982831 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfcd90-994f-4e59-9c06-11cd0c759997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:50.983460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfcd90-994f-4e59-9c06-11cd0c759997 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:50.983736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfcd90-994f-4e59-9c06-11cd0c759997 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:50.984043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfcd90-994f-4e59-9c06-11cd0c759997 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:50.985164 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfcd90-994f-4e59-9c06-11cd0c759997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:50.986604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfcd90-994f-4e59-9c06-11cd0c759997 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:50.991514 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 511/1026] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:50.993985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414125ef-6db4-4a81-ab90-da3298946580 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:51.095789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414125ef-6db4-4a81-ab90-da3298946580 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:51.105263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:51.130265 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c11b5-8f7f-430f-b581-2c360dd1956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:51.130864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c11b5-8f7f-430f-b581-2c360dd1956f None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:51.131139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c11b5-8f7f-430f-b581-2c360dd1956f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:51.131409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c11b5-8f7f-430f-b581-2c360dd1956f None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:51.132306 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c11b5-8f7f-430f-b581-2c360dd1956f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:51.134005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c11b5-8f7f-430f-b581-2c360dd1956f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:51.214946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 516/1027] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:06:50 2022] POST /identity/v3/projects => generated 311 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:06:51.372724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b33a13-f232-4148-ae1f-5ae7dffdf507 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:51.373068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b33a13-f232-4148-ae1f-5ae7dffdf507 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:51.373252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b33a13-f232-4148-ae1f-5ae7dffdf507 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:51.374702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 512/1028] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:51.379831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd256e6-6a3a-4692-bc7a-e5399b81ffe5 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:51.380122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd256e6-6a3a-4692-bc7a-e5399b81ffe5 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:51.380426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd256e6-6a3a-4692-bc7a-e5399b81ffe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:51.400337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7bd256e6-6a3a-4692-bc7a-e5399b81ffe5 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:51.401599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7bd256e6-6a3a-4692-bc7a-e5399b81ffe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:51.473682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 517/1029] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:51.748609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b33a13-f232-4148-ae1f-5ae7dffdf507 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:51.855539 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b33a13-f232-4148-ae1f-5ae7dffdf507 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:51.859471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:51.891337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9042-fa31-4db2-b82d-11b52237f093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:51.892014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9042-fa31-4db2-b82d-11b52237f093 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:51.892344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9042-fa31-4db2-b82d-11b52237f093 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:51.892697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9042-fa31-4db2-b82d-11b52237f093 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:51.904571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e9042-fa31-4db2-b82d-11b52237f093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-e34e9042-fa31-4db2-b82d-11b52237f093 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:51.909529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:51.910565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:51.911418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:51.911418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:51.911418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:51.911418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:51.911418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:51.911418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:51.911418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:51.911418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:51.911418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:51.911418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:51.911418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:06:51.911418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:51.911418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 513/1030] 10.222.0.85 () {56 vars in 1120 bytes} [Mon Aug 29 20:06:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:51.918200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd256e6-6a3a-4692-bc7a-e5399b81ffe5 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:52.013849 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:52.024290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd256e6-6a3a-4692-bc7a-e5399b81ffe5 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:52.059053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929daebb-6fd0-4dfd-9443-1ed641c84429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:52.059703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929daebb-6fd0-4dfd-9443-1ed641c84429 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:52.060888 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929daebb-6fd0-4dfd-9443-1ed641c84429 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:52.061219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929daebb-6fd0-4dfd-9443-1ed641c84429 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:52.062882 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929daebb-6fd0-4dfd-9443-1ed641c84429 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:52.063206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929daebb-6fd0-4dfd-9443-1ed641c84429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:52.064607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929daebb-6fd0-4dfd-9443-1ed641c84429 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:52.078168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 518/1031] 10.222.0.85 () {56 vars in 1115 bytes} [Mon Aug 29 20:06:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:52.079697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9042-fa31-4db2-b82d-11b52237f093 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:52.196029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9042-fa31-4db2-b82d-11b52237f093 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:52.232895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a297380-3581-44f7-9cfc-10f7b5199884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:52.233910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a297380-3581-44f7-9cfc-10f7b5199884 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:52.234395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a297380-3581-44f7-9cfc-10f7b5199884 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:52.234878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a297380-3581-44f7-9cfc-10f7b5199884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:52.247153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a297380-3581-44f7-9cfc-10f7b5199884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:52.249051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a297380-3581-44f7-9cfc-10f7b5199884 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-3a297380-3581-44f7-9cfc-10f7b5199884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:52.265771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:52.266758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:52.267670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:52.267670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:52.267670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:52.275773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929daebb-6fd0-4dfd-9443-1ed641c84429 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:52.276930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 514/1032] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:52.378497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8694-e2e1-415f-9b7a-55234cbe739f None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:52.378777 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8694-e2e1-415f-9b7a-55234cbe739f None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:52.379070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8694-e2e1-415f-9b7a-55234cbe739f None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:52.380428 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 515/1033] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:06:52.385307 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedfd5c-b560-428f-9980-992c5785f344 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:52.385596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedfd5c-b560-428f-9980-992c5785f344 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:52.385870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedfd5c-b560-428f-9980-992c5785f344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:52.393088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929daebb-6fd0-4dfd-9443-1ed641c84429 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:52.408188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-cdedfd5c-b560-428f-9980-992c5785f344 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:52.409370 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cdedfd5c-b560-428f-9980-992c5785f344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:52.421755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:52.441647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f8372-57d8-4950-bd86-e95c1a538e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:52.442806 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f8372-57d8-4950-bd86-e95c1a538e0f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:52.443421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f8372-57d8-4950-bd86-e95c1a538e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:52.443939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f8372-57d8-4950-bd86-e95c1a538e0f None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:52.444838 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f8372-57d8-4950-bd86-e95c1a538e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:52.445591 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f8372-57d8-4950-bd86-e95c1a538e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:52.447994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f8372-57d8-4950-bd86-e95c1a538e0f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:52.471471 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 519/1034] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:52.473687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f8372-57d8-4950-bd86-e95c1a538e0f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:52.489256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 516/1035] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:52.493956 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:52.588956 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f8372-57d8-4950-bd86-e95c1a538e0f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:52.632643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f130dc-36f2-44f0-a4c8-84d4dc355baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:52.633276 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f130dc-36f2-44f0-a4c8-84d4dc355baf None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:52.633560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f130dc-36f2-44f0-a4c8-84d4dc355baf None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:52.633834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f130dc-36f2-44f0-a4c8-84d4dc355baf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:52.642393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f130dc-36f2-44f0-a4c8-84d4dc355baf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:52.643848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f130dc-36f2-44f0-a4c8-84d4dc355baf None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-92f130dc-36f2-44f0-a4c8-84d4dc355baf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:52.656828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:52.657878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:52.663990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:52.663990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:52.663990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:52.663990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:52.663990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:52.663990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:52.663990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:52.663990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:52.663990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:06:52.663990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:52.663990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 520/1036] 10.222.0.85 () {56 vars in 1093 bytes} [Mon Aug 29 20:06:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:52.664917 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdedfd5c-b560-428f-9980-992c5785f344 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:52.761305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f130dc-36f2-44f0-a4c8-84d4dc355baf None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:52.766375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdedfd5c-b560-428f-9980-992c5785f344 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:52.805233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e21d28-3abd-4fad-b8dd-eaea3df08362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:52.805871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e21d28-3abd-4fad-b8dd-eaea3df08362 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:52.806151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e21d28-3abd-4fad-b8dd-eaea3df08362 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:52.806422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e21d28-3abd-4fad-b8dd-eaea3df08362 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:52.807207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e21d28-3abd-4fad-b8dd-eaea3df08362 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:52.807511 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e21d28-3abd-4fad-b8dd-eaea3df08362 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:52.809071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e21d28-3abd-4fad-b8dd-eaea3df08362 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:52.828102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 517/1037] 10.222.0.85 () {56 vars in 1133 bytes} [Mon Aug 29 20:06:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:52.833808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e21d28-3abd-4fad-b8dd-eaea3df08362 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:52.838391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:52.851029 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:52.886346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f130dc-36f2-44f0-a4c8-84d4dc355baf None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:52.927366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face7f01-0062-43a0-a849-552bef94f476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:52.927995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face7f01-0062-43a0-a849-552bef94f476 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:52.928299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face7f01-0062-43a0-a849-552bef94f476 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:52.928834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face7f01-0062-43a0-a849-552bef94f476 None admin] PATH_INFO: `/v3/services/image` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:52.929147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face7f01-0062-43a0-a849-552bef94f476 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:52.930504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face7f01-0062-43a0-a849-552bef94f476 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:52.938687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e21d28-3abd-4fad-b8dd-eaea3df08362 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-face7f01-0062-43a0-a849-552bef94f476 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:52.939850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:52.948637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:52.949601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 521/1038] 10.222.0.85 () {56 vars in 1105 bytes} [Mon Aug 29 20:06:52 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:52.950431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face7f01-0062-43a0-a849-552bef94f476 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:52.979444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb0947-f89b-496d-ac5f-ded6e3c3e8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:52.980143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb0947-f89b-496d-ac5f-ded6e3c3e8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:52.980527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb0947-f89b-496d-ac5f-ded6e3c3e8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:52.980860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb0947-f89b-496d-ac5f-ded6e3c3e8b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:53.058895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb0947-f89b-496d-ac5f-ded6e3c3e8b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:53.060899 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb0947-f89b-496d-ac5f-ded6e3c3e8b3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:53.062395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face7f01-0062-43a0-a849-552bef94f476 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:53.063819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:53.100053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e794b44-4b52-4334-a872-5f8bd87e1c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:53.100831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e794b44-4b52-4334-a872-5f8bd87e1c9f None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:53.101238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e794b44-4b52-4334-a872-5f8bd87e1c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:53.101549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e794b44-4b52-4334-a872-5f8bd87e1c9f None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:53.102251 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e794b44-4b52-4334-a872-5f8bd87e1c9f None admin] RBAC: Adding query filter params (name=image) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:53.102623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e794b44-4b52-4334-a872-5f8bd87e1c9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:53.104063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e794b44-4b52-4334-a872-5f8bd87e1c9f None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:53.106067 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:53.114845 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 522/1039] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:53.116040 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e794b44-4b52-4334-a872-5f8bd87e1c9f None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:53.180098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 518/1040] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:52 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 20:06:53.181191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb0947-f89b-496d-ac5f-ded6e3c3e8b3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:53.226683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e794b44-4b52-4334-a872-5f8bd87e1c9f None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:53.260766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd78de2-69f6-4459-8ee6-c19cdf55f866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:53.261376 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd78de2-69f6-4459-8ee6-c19cdf55f866 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:53.261648 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd78de2-69f6-4459-8ee6-c19cdf55f866 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:53.261924 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd78de2-69f6-4459-8ee6-c19cdf55f866 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:53.262613 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd78de2-69f6-4459-8ee6-c19cdf55f866 None admin] RBAC: Adding query filter params (type=image) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:53.262912 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd78de2-69f6-4459-8ee6-c19cdf55f866 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:53.264914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd78de2-69f6-4459-8ee6-c19cdf55f866 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:53.269721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 523/1041] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:53.269721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c1b1f-1267-4793-936e-bdcc1dcf754e None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:53.269721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c1b1f-1267-4793-936e-bdcc1dcf754e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:53.269721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c1b1f-1267-4793-936e-bdcc1dcf754e None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:53.269721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 524/1042] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:53.277562 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c1b1f-1267-4793-936e-bdcc1dcf754e None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:53.282356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb0947-f89b-496d-ac5f-ded6e3c3e8b3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:53.284750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:53.317733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b27e6-ab6b-499c-81a8-4507754cc361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:53.318364 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b27e6-ab6b-499c-81a8-4507754cc361 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:53.318646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b27e6-ab6b-499c-81a8-4507754cc361 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:53.318921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b27e6-ab6b-499c-81a8-4507754cc361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:53.329120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b27e6-ab6b-499c-81a8-4507754cc361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:53.330562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b27e6-ab6b-499c-81a8-4507754cc361 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:53.340561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-870b27e6-ab6b-499c-81a8-4507754cc361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:53.347361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:53.353371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:53.354553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:53.354553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:53.354553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:53.354553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 519/1043] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:53.354553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cd1b6-90ae-4e59-bbf6-6e6b163ad4db None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:53.354553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cd1b6-90ae-4e59-bbf6-6e6b163ad4db None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:53.354553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cd1b6-90ae-4e59-bbf6-6e6b163ad4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:53.370774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d44cd1b6-90ae-4e59-bbf6-6e6b163ad4db None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:53.371970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d44cd1b6-90ae-4e59-bbf6-6e6b163ad4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:53.380542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c1b1f-1267-4793-936e-bdcc1dcf754e None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:53.416387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a7a38-e0e0-4c92-8ea9-8701420adad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:53.417136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a7a38-e0e0-4c92-8ea9-8701420adad0 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:53.417396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a7a38-e0e0-4c92-8ea9-8701420adad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:53.417744 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a7a38-e0e0-4c92-8ea9-8701420adad0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:53.418359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a7a38-e0e0-4c92-8ea9-8701420adad0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:53.419964 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a7a38-e0e0-4c92-8ea9-8701420adad0 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:53.438385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 520/1044] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:53.439231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cd1b6-90ae-4e59-bbf6-6e6b163ad4db None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:53.510775 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 525/1045] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:06:53 2022] POST /identity/v3/endpoints => generated 327 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:06:53.534112 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:53.550794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cd1b6-90ae-4e59-bbf6-6e6b163ad4db None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:53.597036 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616315ce-5a9c-4e83-b9bc-d3a8d43518da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:53.597671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616315ce-5a9c-4e83-b9bc-d3a8d43518da None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:53.597939 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616315ce-5a9c-4e83-b9bc-d3a8d43518da None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:53.598208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616315ce-5a9c-4e83-b9bc-d3a8d43518da None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:53.598882 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616315ce-5a9c-4e83-b9bc-d3a8d43518da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:53.599159 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616315ce-5a9c-4e83-b9bc-d3a8d43518da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:53.600571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616315ce-5a9c-4e83-b9bc-d3a8d43518da None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:53.624840 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 521/1046] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:53.626139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a7a38-e0e0-4c92-8ea9-8701420adad0 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:53.706496 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616315ce-5a9c-4e83-b9bc-d3a8d43518da None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:53.736683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a7a38-e0e0-4c92-8ea9-8701420adad0 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:53.760590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:53.775926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be6ee2-04d5-4d65-996b-1073e5b9c7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:53.776725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be6ee2-04d5-4d65-996b-1073e5b9c7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:53.776875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be6ee2-04d5-4d65-996b-1073e5b9c7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:53.783850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be6ee2-04d5-4d65-996b-1073e5b9c7f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:53.785114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be6ee2-04d5-4d65-996b-1073e5b9c7f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:53.786560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be6ee2-04d5-4d65-996b-1073e5b9c7f2 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-66be6ee2-04d5-4d65-996b-1073e5b9c7f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:53.798399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:53.799320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:53.800249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:53.800249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:53.800249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:53.800249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:53.800249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:53.800249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:53.800249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:53.800249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:53.800249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 526/1047] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:53.805323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be6ee2-04d5-4d65-996b-1073e5b9c7f2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:53.818041 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616315ce-5a9c-4e83-b9bc-d3a8d43518da None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:53.854182 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acd233-cca2-424b-ae36-d39bc08b14a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:53.854672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acd233-cca2-424b-ae36-d39bc08b14a0 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:53.854953 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acd233-cca2-424b-ae36-d39bc08b14a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:53.855221 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acd233-cca2-424b-ae36-d39bc08b14a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:53.856639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39acd233-cca2-424b-ae36-d39bc08b14a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:53.858686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39acd233-cca2-424b-ae36-d39bc08b14a0 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:53.860214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 522/1048] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:53.865699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acd233-cca2-424b-ae36-d39bc08b14a0 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:53.879585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:53.913091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be6ee2-04d5-4d65-996b-1073e5b9c7f2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:53.947492 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509ebef-8719-459c-b67a-21c8f26239d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:53.948102 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509ebef-8719-459c-b67a-21c8f26239d5 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:53.948420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509ebef-8719-459c-b67a-21c8f26239d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:53.948689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509ebef-8719-459c-b67a-21c8f26239d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:53.949407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509ebef-8719-459c-b67a-21c8f26239d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:53.949718 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509ebef-8719-459c-b67a-21c8f26239d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:53.951693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509ebef-8719-459c-b67a-21c8f26239d5 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:53.965349 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 527/1049] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:53.971362 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509ebef-8719-459c-b67a-21c8f26239d5 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:53.980993 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acd233-cca2-424b-ae36-d39bc08b14a0 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:54.015772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:54.019148 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfd4de-36ed-4744-ac2d-ea3deaa035af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:54.019776 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfd4de-36ed-4744-ac2d-ea3deaa035af None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:54.020051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfd4de-36ed-4744-ac2d-ea3deaa035af None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:54.020353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfd4de-36ed-4744-ac2d-ea3deaa035af None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:54.021008 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbfd4de-36ed-4744-ac2d-ea3deaa035af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:54.022448 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbfd4de-36ed-4744-ac2d-ea3deaa035af None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:54.074947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509ebef-8719-459c-b67a-21c8f26239d5 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:54.110240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b70a8-9441-47dc-a1ce-08f2b1774035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:54.110851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b70a8-9441-47dc-a1ce-08f2b1774035 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:54.111397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b70a8-9441-47dc-a1ce-08f2b1774035 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:54.111507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 523/1050] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:06:53 2022] POST /identity/v3/users => generated 285 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:06:54.111588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b70a8-9441-47dc-a1ce-08f2b1774035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:54.112258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b70a8-9441-47dc-a1ce-08f2b1774035 None admin] RBAC: Adding query filter params (role.id=130698a8e0c6468491e7d81752709939, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=805df3047fee45c7a702d72a5c082eb5) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:54.112515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b70a8-9441-47dc-a1ce-08f2b1774035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:54.114033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b70a8-9441-47dc-a1ce-08f2b1774035 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:54.129364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 528/1051] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:53 2022] GET /identity/v3/role_assignments?user.id=805df3047fee45c7a702d72a5c082eb5&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=130698a8e0c6468491e7d81752709939 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:55.460342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6591b-f02e-4ed9-a623-3fb5bd253a76 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:55.460918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6591b-f02e-4ed9-a623-3fb5bd253a76 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:55.460918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6591b-f02e-4ed9-a623-3fb5bd253a76 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:55.462215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 524/1052] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:55.467416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2ba5d-b236-4257-aa75-bd7dd47eec28 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:55.467831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2ba5d-b236-4257-aa75-bd7dd47eec28 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:55.468016 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2ba5d-b236-4257-aa75-bd7dd47eec28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:55.486859 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-07a2ba5d-b236-4257-aa75-bd7dd47eec28 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:55.488035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-07a2ba5d-b236-4257-aa75-bd7dd47eec28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:55.560787 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 529/1053] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:55.830654 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6591b-f02e-4ed9-a623-3fb5bd253a76 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:55.902071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:55.945521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6591b-f02e-4ed9-a623-3fb5bd253a76 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:55.984751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67177c5-a14c-46d8-8f0e-a8c7de0e380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:55.985363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67177c5-a14c-46d8-8f0e-a8c7de0e380c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:55.985640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67177c5-a14c-46d8-8f0e-a8c7de0e380c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:55.985909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67177c5-a14c-46d8-8f0e-a8c7de0e380c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:56.060758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67177c5-a14c-46d8-8f0e-a8c7de0e380c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:56.062204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67177c5-a14c-46d8-8f0e-a8c7de0e380c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:56.085065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f9697-be00-4936-bd21-4c74fc602af8 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:56.085348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f9697-be00-4936-bd21-4c74fc602af8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:56.085542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f9697-be00-4936-bd21-4c74fc602af8 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:56.086882 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 530/1054] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:56.091860 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4a30-59a7-41a0-8fce-545d4f718390 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:56.092144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4a30-59a7-41a0-8fce-545d4f718390 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:56.092449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4a30-59a7-41a0-8fce-545d4f718390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:56.111132 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-d83c4a30-59a7-41a0-8fce-545d4f718390 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:56.112359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-d83c4a30-59a7-41a0-8fce-545d4f718390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:56.132345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:56.140909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:56.170981 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 525/1055] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:06:56.172324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bc1c8-e33e-42a3-9bf1-fa793569a528 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:56.172615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bc1c8-e33e-42a3-9bf1-fa793569a528 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:56.172891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bc1c8-e33e-42a3-9bf1-fa793569a528 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:56.174147 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 526/1056] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:56.177204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3bc1c8-e33e-42a3-9bf1-fa793569a528 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:56.183645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 531/1057] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:06:56.184928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4d96c-dde9-45e2-bd1f-3b5d781f16e5 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:56.185220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4d96c-dde9-45e2-bd1f-3b5d781f16e5 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:56.185488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4d96c-dde9-45e2-bd1f-3b5d781f16e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:56.207586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-02c4d96c-dde9-45e2-bd1f-3b5d781f16e5 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:56.208968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-02c4d96c-dde9-45e2-bd1f-3b5d781f16e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:56.288753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 532/1058] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:56.290486 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3bc1c8-e33e-42a3-9bf1-fa793569a528 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:56.329376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3d398-d024-4ecb-a24f-f98648d20442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:56.330007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3d398-d024-4ecb-a24f-f98648d20442 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:56.330283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3d398-d024-4ecb-a24f-f98648d20442 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:56.330556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3d398-d024-4ecb-a24f-f98648d20442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:56.342085 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3d398-d024-4ecb-a24f-f98648d20442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:56.343492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3d398-d024-4ecb-a24f-f98648d20442 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-32a3d398-d024-4ecb-a24f-f98648d20442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:56.360477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:56.361488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:56.363038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:56.363038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:56.363038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:56.363038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 527/1059] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:56.367807 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4d96c-dde9-45e2-bd1f-3b5d781f16e5 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:56.456176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3d398-d024-4ecb-a24f-f98648d20442 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:56.477268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:56.477913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:56.485180 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4d96c-dde9-45e2-bd1f-3b5d781f16e5 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:56.531851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94772edf-9c49-4362-8041-4eac6473cc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:56.532521 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94772edf-9c49-4362-8041-4eac6473cc0d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:56.532804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94772edf-9c49-4362-8041-4eac6473cc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:56.533081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94772edf-9c49-4362-8041-4eac6473cc0d None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:56.533831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94772edf-9c49-4362-8041-4eac6473cc0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:56.534140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94772edf-9c49-4362-8041-4eac6473cc0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:56.535500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94772edf-9c49-4362-8041-4eac6473cc0d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:56.550678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 533/1060] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:56.559297 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94772edf-9c49-4362-8041-4eac6473cc0d None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:56.572938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3d398-d024-4ecb-a24f-f98648d20442 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:56.624871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0019c9a-4c96-4a7d-8dd9-2d127da5a73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:56.625793 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0019c9a-4c96-4a7d-8dd9-2d127da5a73f None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:56.626932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0019c9a-4c96-4a7d-8dd9-2d127da5a73f None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:56.627367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0019c9a-4c96-4a7d-8dd9-2d127da5a73f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:56.645729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0019c9a-4c96-4a7d-8dd9-2d127da5a73f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-d0019c9a-4c96-4a7d-8dd9-2d127da5a73f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:56.652803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:56.653862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:56.657047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:56.657047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:56.657047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:56.657047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:56.657047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:56.657047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:56.657047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:56.657047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:56.657047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:56.657047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:56.657047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:06:56.657047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:56.657047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 528/1061] 10.222.0.85 () {56 vars in 1120 bytes} [Mon Aug 29 20:06:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:56.661357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0019c9a-4c96-4a7d-8dd9-2d127da5a73f None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:56.688961 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94772edf-9c49-4362-8041-4eac6473cc0d None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:56.734949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa466f89-f5f2-48b4-b9c9-087b386774c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:56.735591 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa466f89-f5f2-48b4-b9c9-087b386774c7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:56.735870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa466f89-f5f2-48b4-b9c9-087b386774c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:56.736144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa466f89-f5f2-48b4-b9c9-087b386774c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:56.751045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa466f89-f5f2-48b4-b9c9-087b386774c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-aa466f89-f5f2-48b4-b9c9-087b386774c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:06:56.756113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:56.757151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:56.758005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:56.758005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:56.758005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:56.758005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:56.758005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:06:56.758005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:06:56.758005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:06:56.758005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:06:56.758005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:06:56.758005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:06:56.758005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:06:56.758005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:56.758648 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 534/1062] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:06:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:56.765173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa466f89-f5f2-48b4-b9c9-087b386774c7 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:56.769895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:56.783872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:56.797079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0019c9a-4c96-4a7d-8dd9-2d127da5a73f None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:56.836853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdbd6b-d166-4b39-848b-14b59c5f7ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:56.837914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdbd6b-d166-4b39-848b-14b59c5f7ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:56.838242 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdbd6b-d166-4b39-848b-14b59c5f7ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:56.838519 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdbd6b-d166-4b39-848b-14b59c5f7ec8 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:56.839197 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdbd6b-d166-4b39-848b-14b59c5f7ec8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:56.839501 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdbd6b-d166-4b39-848b-14b59c5f7ec8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:56.841023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdbd6b-d166-4b39-848b-14b59c5f7ec8 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:56.850729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 529/1063] 10.222.0.85 () {56 vars in 1115 bytes} [Mon Aug 29 20:06:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:56.856861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdbd6b-d166-4b39-848b-14b59c5f7ec8 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:56.880020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa466f89-f5f2-48b4-b9c9-087b386774c7 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:56.915426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339848df-f45a-42c4-be33-d7b321671f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:56.916404 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339848df-f45a-42c4-be33-d7b321671f70 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:56.916891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339848df-f45a-42c4-be33-d7b321671f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:56.917257 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339848df-f45a-42c4-be33-d7b321671f70 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:56.917940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339848df-f45a-42c4-be33-d7b321671f70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:56.918245 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339848df-f45a-42c4-be33-d7b321671f70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:56.920378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339848df-f45a-42c4-be33-d7b321671f70 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:56.928867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 535/1064] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:06:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:56.934647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339848df-f45a-42c4-be33-d7b321671f70 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:56.966388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdbd6b-d166-4b39-848b-14b59c5f7ec8 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:57.001218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ad36a-12ff-4177-a4ac-1090632dd896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:57.001836 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ad36a-12ff-4177-a4ac-1090632dd896 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:57.002114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ad36a-12ff-4177-a4ac-1090632dd896 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:57.002380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ad36a-12ff-4177-a4ac-1090632dd896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:57.014008 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:57.016408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ad36a-12ff-4177-a4ac-1090632dd896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:57.017070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:57.017857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ad36a-12ff-4177-a4ac-1090632dd896 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-fa1ad36a-12ff-4177-a4ac-1090632dd896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:57.032547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:57.033815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:57.034835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:57.034835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:57.034835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:57.034835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 530/1065] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:57.039374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ad36a-12ff-4177-a4ac-1090632dd896 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:57.040854 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339848df-f45a-42c4-be33-d7b321671f70 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:57.074528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb7633-4bdc-497f-a1a5-58f1624daf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:57.075212 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb7633-4bdc-497f-a1a5-58f1624daf23 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:57.076449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb7633-4bdc-497f-a1a5-58f1624daf23 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:57.076912 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb7633-4bdc-497f-a1a5-58f1624daf23 None admin] PATH_INFO: `/v3/users/859784a24733408f8aac3ad9537602e9` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:57.090424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb7633-4bdc-497f-a1a5-58f1624daf23 None admin] RBAC: Authorizing `identity:get_user(user_id=859784a24733408f8aac3ad9537602e9)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:57.091875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb7633-4bdc-497f-a1a5-58f1624daf23 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:57.093880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 536/1066] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:06:56 2022] GET /identity/v3/users/859784a24733408f8aac3ad9537602e9 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:57.101117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb7633-4bdc-497f-a1a5-58f1624daf23 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:57.142547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ad36a-12ff-4177-a4ac-1090632dd896 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:57.176615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc7aa8-311e-478d-8d2e-5ad2c346a015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:57.177368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc7aa8-311e-478d-8d2e-5ad2c346a015 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:57.177528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc7aa8-311e-478d-8d2e-5ad2c346a015 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:57.177803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc7aa8-311e-478d-8d2e-5ad2c346a015 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:57.178430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc7aa8-311e-478d-8d2e-5ad2c346a015 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:57.178730 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc7aa8-311e-478d-8d2e-5ad2c346a015 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:57.180089 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc7aa8-311e-478d-8d2e-5ad2c346a015 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:57.193864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 531/1067] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:57.199400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc7aa8-311e-478d-8d2e-5ad2c346a015 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:57.205809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb7633-4bdc-497f-a1a5-58f1624daf23 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:57.241032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb409c0-fa69-4d57-b1d2-0fe35fff084e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:57.241686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb409c0-fa69-4d57-b1d2-0fe35fff084e None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:57.242009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb409c0-fa69-4d57-b1d2-0fe35fff084e None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:57.242267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb409c0-fa69-4d57-b1d2-0fe35fff084e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:57.245425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:57.253997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:57.301513 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc7aa8-311e-478d-8d2e-5ad2c346a015 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:57.314727 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb409c0-fa69-4d57-b1d2-0fe35fff084e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:57.316195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb409c0-fa69-4d57-b1d2-0fe35fff084e None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:57.337474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e028e-7614-4c6b-81d8-3fff81254366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:57.338101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e028e-7614-4c6b-81d8-3fff81254366 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:57.338382 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e028e-7614-4c6b-81d8-3fff81254366 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:57.338761 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e028e-7614-4c6b-81d8-3fff81254366 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:57.346555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e028e-7614-4c6b-81d8-3fff81254366 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:57.347993 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e028e-7614-4c6b-81d8-3fff81254366 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-930e028e-7614-4c6b-81d8-3fff81254366 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:06:57.360255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:57.361453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:57.362862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:57.362862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:57.362862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:06:57.362862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:06:57.362862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:06:57.362862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:06:57.362862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:06:57.362862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:06:57.362862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:06:57.362862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:57.364317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 532/1068] 10.222.0.85 () {56 vars in 1093 bytes} [Mon Aug 29 20:06:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:57.367318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e028e-7614-4c6b-81d8-3fff81254366 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:57.421158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 537/1069] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:06:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 20:06:57.427044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb409c0-fa69-4d57-b1d2-0fe35fff084e None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:57.460822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:57.471894 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e028e-7614-4c6b-81d8-3fff81254366 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:57.486265 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:57.506999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e8b4d-8bce-452f-8b40-136628bfd0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:57.507611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e8b4d-8bce-452f-8b40-136628bfd0aa None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:57.507889 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e8b4d-8bce-452f-8b40-136628bfd0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:57.508250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e8b4d-8bce-452f-8b40-136628bfd0aa None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:57.508980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e8b4d-8bce-452f-8b40-136628bfd0aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:57.509289 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e8b4d-8bce-452f-8b40-136628bfd0aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:57.510868 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e8b4d-8bce-452f-8b40-136628bfd0aa None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:57.526282 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb409c0-fa69-4d57-b1d2-0fe35fff084e None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:57.529631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 533/1070] 10.222.0.85 () {56 vars in 1133 bytes} [Mon Aug 29 20:06:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:57.535274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e8b4d-8bce-452f-8b40-136628bfd0aa None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:57.561521 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984baed-a36b-4586-a649-2933106f83d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:57.562154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984baed-a36b-4586-a649-2933106f83d8 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:57.562429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984baed-a36b-4586-a649-2933106f83d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:57.562700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984baed-a36b-4586-a649-2933106f83d8 None admin] PATH_INFO: `/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:57.563815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984baed-a36b-4586-a649-2933106f83d8 None admin] RBAC: Authorizing `identity:get_project(project_id=efc7b01f484943d3bc4eac2e07a81d4d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:57.565278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984baed-a36b-4586-a649-2933106f83d8 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:57.566910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 538/1071] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:06:57 2022] GET /identity/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:57.573103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984baed-a36b-4586-a649-2933106f83d8 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:57.638816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e8b4d-8bce-452f-8b40-136628bfd0aa None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:57.675077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fd5cf-36ef-4b0d-92ff-66a931913cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:57.675490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984baed-a36b-4586-a649-2933106f83d8 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:57.675627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fd5cf-36ef-4b0d-92ff-66a931913cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:57.675908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fd5cf-36ef-4b0d-92ff-66a931913cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:57.676204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fd5cf-36ef-4b0d-92ff-66a931913cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:57.677068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:57.687369 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fd5cf-36ef-4b0d-92ff-66a931913cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:57.688567 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fd5cf-36ef-4b0d-92ff-66a931913cb1 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-456fd5cf-36ef-4b0d-92ff-66a931913cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:06:57.704093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:57.705189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:06:57.706644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:06:57.706644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:06:57.706644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:06:57.706644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 534/1072] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:06:57.711048 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fd5cf-36ef-4b0d-92ff-66a931913cb1 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:57.711243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9079b35-ecc8-4f9f-9230-6cb89ffa80e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:57.711843 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9079b35-ecc8-4f9f-9230-6cb89ffa80e6 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:57.712119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9079b35-ecc8-4f9f-9230-6cb89ffa80e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:57.712432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9079b35-ecc8-4f9f-9230-6cb89ffa80e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:57.713148 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9079b35-ecc8-4f9f-9230-6cb89ffa80e6 None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=efc7b01f484943d3bc4eac2e07a81d4d, user.id=859784a24733408f8aac3ad9537602e9) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:57.713452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9079b35-ecc8-4f9f-9230-6cb89ffa80e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:57.714869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9079b35-ecc8-4f9f-9230-6cb89ffa80e6 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:57.721102 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 539/1073] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:06:57 2022] GET /identity/v3/role_assignments?user.id=859784a24733408f8aac3ad9537602e9&scope.project.id=efc7b01f484943d3bc4eac2e07a81d4d&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:57.743732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:57.826411 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fd5cf-36ef-4b0d-92ff-66a931913cb1 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:57.861283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d291c-a6a8-44b7-a76f-2afe1ec094fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:57.861898 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d291c-a6a8-44b7-a76f-2afe1ec094fb None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:57.862661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d291c-a6a8-44b7-a76f-2afe1ec094fb None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:57.862661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d291c-a6a8-44b7-a76f-2afe1ec094fb None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:57.863082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d291c-a6a8-44b7-a76f-2afe1ec094fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:57.863419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d291c-a6a8-44b7-a76f-2afe1ec094fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:57.865841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d291c-a6a8-44b7-a76f-2afe1ec094fb None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:57.880615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 535/1074] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:06:57.886260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9079b35-ecc8-4f9f-9230-6cb89ffa80e6 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:57.987038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9079b35-ecc8-4f9f-9230-6cb89ffa80e6 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:58.021611 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43543dc2-f37b-46ec-b1a1-2345faf8c895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:58.022241 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43543dc2-f37b-46ec-b1a1-2345faf8c895 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:58.022518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43543dc2-f37b-46ec-b1a1-2345faf8c895 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:58.022788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43543dc2-f37b-46ec-b1a1-2345faf8c895 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:58.029679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43543dc2-f37b-46ec-b1a1-2345faf8c895 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:58.031103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43543dc2-f37b-46ec-b1a1-2345faf8c895 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-43543dc2-f37b-46ec-b1a1-2345faf8c895 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:06:58.042154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:06:58.043168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:06:58.044214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:06:58.044214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:06:58.044214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:06:58.044214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:06:58.044214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:06:58.044214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:06:58.044214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:06:58.044214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:06:58.044214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 540/1075] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:06:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:06:58.049116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d291c-a6a8-44b7-a76f-2afe1ec094fb None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:58.129430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:58.147142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d291c-a6a8-44b7-a76f-2afe1ec094fb None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:58.181221 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd340b67-ecc7-4748-882f-53a8d4b9352a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:58.181831 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd340b67-ecc7-4748-882f-53a8d4b9352a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:58.182109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd340b67-ecc7-4748-882f-53a8d4b9352a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:58.182379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd340b67-ecc7-4748-882f-53a8d4b9352a None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:58.183223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd340b67-ecc7-4748-882f-53a8d4b9352a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:06:58.183532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd340b67-ecc7-4748-882f-53a8d4b9352a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:58.185596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd340b67-ecc7-4748-882f-53a8d4b9352a None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:58.197024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 536/1076] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:06:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:06:58.203030 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43543dc2-f37b-46ec-b1a1-2345faf8c895 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:06:58.231115 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:06:58.304080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43543dc2-f37b-46ec-b1a1-2345faf8c895 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:06:58.338545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf2da5-5edd-4836-aaa9-b5cd7ef286af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:06:58.339209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf2da5-5edd-4836-aaa9-b5cd7ef286af None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:58.339482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf2da5-5edd-4836-aaa9-b5cd7ef286af None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:58.339755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf2da5-5edd-4836-aaa9-b5cd7ef286af None admin] PATH_INFO: `/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/805df3047fee45c7a702d72a5c082eb5/roles/130698a8e0c6468491e7d81752709939` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:58.341945 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf2da5-5edd-4836-aaa9-b5cd7ef286af None admin] RBAC: Authorizing `identity:create_grant(project_id=11372fe8994543b4a19773c0e0bee53e, user_id=805df3047fee45c7a702d72a5c082eb5, role_id=130698a8e0c6468491e7d81752709939)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:06:58.343382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf2da5-5edd-4836-aaa9-b5cd7ef286af None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:06:58.436816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 541/1077] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:06:58 2022] PUT /identity/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/805df3047fee45c7a702d72a5c082eb5/roles/130698a8e0c6468491e7d81752709939 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:06:59.637601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e8f2b-0c48-4387-9ee1-29d6c181f82c None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:59.638610 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e8f2b-0c48-4387-9ee1-29d6c181f82c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:59.638736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e8f2b-0c48-4387-9ee1-29d6c181f82c None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:59.640015 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 537/1078] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:06:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:06:59.645426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da826f-8235-4354-b642-80c06c5f5579 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:06:59.645820 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da826f-8235-4354-b642-80c06c5f5579 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:06:59.645982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da826f-8235-4354-b642-80c06c5f5579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:06:59.664923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-58da826f-8235-4354-b642-80c06c5f5579 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:06:59.666096 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-58da826f-8235-4354-b642-80c06c5f5579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:06:59.732011 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 542/1079] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:06:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:06:59.995072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e8f2b-0c48-4387-9ee1-29d6c181f82c None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:00.101633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e8f2b-0c48-4387-9ee1-29d6c181f82c None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:00.136903 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8759b-9e1f-4283-a743-a72a5c4dd4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:00.137620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8759b-9e1f-4283-a743-a72a5c4dd4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:00.137904 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8759b-9e1f-4283-a743-a72a5c4dd4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:00.138237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8759b-9e1f-4283-a743-a72a5c4dd4d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:00.149132 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8759b-9e1f-4283-a743-a72a5c4dd4d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-54b8759b-9e1f-4283-a743-a72a5c4dd4d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:07:00.154879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:00.155977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:00.156885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:00.156885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:00.156885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:00.156885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:00.156885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:07:00.156885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:07:00.156885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:07:00.156885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:07:00.156885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:07:00.156885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:07:00.156885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:00.156885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:07:00.156885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 538/1080] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:06:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:07:00.162224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da826f-8235-4354-b642-80c06c5f5579 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:00.180198 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:00.261761 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da826f-8235-4354-b642-80c06c5f5579 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:00.295536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e128a35-777f-490d-b26c-422e29cfd0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:00.296135 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e128a35-777f-490d-b26c-422e29cfd0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:00.296435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e128a35-777f-490d-b26c-422e29cfd0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:00.296700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e128a35-777f-490d-b26c-422e29cfd0d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:00.297381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e128a35-777f-490d-b26c-422e29cfd0d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:00.297676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e128a35-777f-490d-b26c-422e29cfd0d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:00.299012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e128a35-777f-490d-b26c-422e29cfd0d8 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:00.307171 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 543/1081] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:07:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:00.313722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08b438-e94a-41ea-9450-fa98627d9cf4 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:00.314097 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08b438-e94a-41ea-9450-fa98627d9cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:00.314446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08b438-e94a-41ea-9450-fa98627d9cf4 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:00.314715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e128a35-777f-490d-b26c-422e29cfd0d8 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:00.315931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 539/1082] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:00.320939 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874c95c-661d-4f57-9e08-e9b5963a5460 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:00.321227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874c95c-661d-4f57-9e08-e9b5963a5460 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:00.321502 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874c95c-661d-4f57-9e08-e9b5963a5460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:00.328493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:00.340955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d874c95c-661d-4f57-9e08-e9b5963a5460 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:00.343775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d874c95c-661d-4f57-9e08-e9b5963a5460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:00.397862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:00.416955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 540/1083] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:07:00.421048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e128a35-777f-490d-b26c-422e29cfd0d8 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:00.455910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a3a76-adfc-42cc-bda4-eff79e9c3ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:00.456565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a3a76-adfc-42cc-bda4-eff79e9c3ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:00.456844 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a3a76-adfc-42cc-bda4-eff79e9c3ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:00.457115 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a3a76-adfc-42cc-bda4-eff79e9c3ae7 None admin] PATH_INFO: `/v3/users/859784a24733408f8aac3ad9537602e9` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:00.458230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a3a76-adfc-42cc-bda4-eff79e9c3ae7 None admin] RBAC: Authorizing `identity:get_user(user_id=859784a24733408f8aac3ad9537602e9)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:00.459647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a3a76-adfc-42cc-bda4-eff79e9c3ae7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:00.461256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 544/1084] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:00 2022] GET /identity/v3/users/859784a24733408f8aac3ad9537602e9 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:07:00.467002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874c95c-661d-4f57-9e08-e9b5963a5460 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:00.573347 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874c95c-661d-4f57-9e08-e9b5963a5460 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:00.604995 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:00.611066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a295b-3d1a-4f76-a217-ff8b3a89510d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:00.611938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a295b-3d1a-4f76-a217-ff8b3a89510d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:00.612252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a295b-3d1a-4f76-a217-ff8b3a89510d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:00.612532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a295b-3d1a-4f76-a217-ff8b3a89510d None admin] PATH_INFO: `/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:00.613703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a295b-3d1a-4f76-a217-ff8b3a89510d None admin] RBAC: Authorizing `identity:get_project(project_id=efc7b01f484943d3bc4eac2e07a81d4d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:00.615128 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a295b-3d1a-4f76-a217-ff8b3a89510d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:00.616779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 541/1085] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:07:00 2022] GET /identity/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:07:00.623268 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a3a76-adfc-42cc-bda4-eff79e9c3ae7 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:00.693856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a295b-3d1a-4f76-a217-ff8b3a89510d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:00.727842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a3a76-adfc-42cc-bda4-eff79e9c3ae7 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:00.762054 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a93987-eac1-4cb0-a13d-a0b741a12def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:00.762700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a93987-eac1-4cb0-a13d-a0b741a12def None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:00.763032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a93987-eac1-4cb0-a13d-a0b741a12def None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:00.763813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a93987-eac1-4cb0-a13d-a0b741a12def None admin] PATH_INFO: `/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d/users/859784a24733408f8aac3ad9537602e9/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:00.765725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a93987-eac1-4cb0-a13d-a0b741a12def None admin] RBAC: Authorizing `identity:create_grant(project_id=efc7b01f484943d3bc4eac2e07a81d4d, user_id=859784a24733408f8aac3ad9537602e9, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:00.767218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a93987-eac1-4cb0-a13d-a0b741a12def None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:00.796540 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a295b-3d1a-4f76-a217-ff8b3a89510d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:00.830637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3a6a7-e6ec-4b50-922b-e0624995c95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:00.831220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3a6a7-e6ec-4b50-922b-e0624995c95f None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:00.831497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3a6a7-e6ec-4b50-922b-e0624995c95f None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:00.831767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3a6a7-e6ec-4b50-922b-e0624995c95f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:00.841736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3a6a7-e6ec-4b50-922b-e0624995c95f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-e8b3a6a7-e6ec-4b50-922b-e0624995c95f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:07:00.846517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:00.847534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:00.849061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:00.849061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:00.849061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:00.849061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:00.849061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:07:00.849061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:07:00.849061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:07:00.849061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:07:00.849061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:07:00.849061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:07:00.849061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:07:00.849061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:07:00.849061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 542/1087] 10.222.0.85 () {56 vars in 1120 bytes} [Mon Aug 29 20:07:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:07:00.852533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 545/1087] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:07:00 2022] PUT /identity/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d/users/859784a24733408f8aac3ad9537602e9/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:07:00.853650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3a6a7-e6ec-4b50-922b-e0624995c95f None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:00.921437 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:00.965598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3a6a7-e6ec-4b50-922b-e0624995c95f None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:01.004613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2ce5b-9244-4455-8c07-6a46ec023d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:01.005218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2ce5b-9244-4455-8c07-6a46ec023d66 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:01.005495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2ce5b-9244-4455-8c07-6a46ec023d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:01.005763 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2ce5b-9244-4455-8c07-6a46ec023d66 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:01.006430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2ce5b-9244-4455-8c07-6a46ec023d66 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:01.006728 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2ce5b-9244-4455-8c07-6a46ec023d66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:01.008094 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2ce5b-9244-4455-8c07-6a46ec023d66 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:01.016492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 543/1088] 10.222.0.85 () {56 vars in 1115 bytes} [Mon Aug 29 20:07:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:01.022504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a93987-eac1-4cb0-a13d-a0b741a12def None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:01.032468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:01.123555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a93987-eac1-4cb0-a13d-a0b741a12def None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:01.156986 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b401c-7518-4395-ad68-6b9eadb3b0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:01.157614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b401c-7518-4395-ad68-6b9eadb3b0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:01.157893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b401c-7518-4395-ad68-6b9eadb3b0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:01.158162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b401c-7518-4395-ad68-6b9eadb3b0d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:01.168555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b401c-7518-4395-ad68-6b9eadb3b0d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:01.170509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b401c-7518-4395-ad68-6b9eadb3b0d6 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-402b401c-7518-4395-ad68-6b9eadb3b0d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:07:01.184378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:01.185547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:07:01.186424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:07:01.186424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:07:01.186424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:07:01.186424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 546/1089] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:07:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:07:01.191410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2ce5b-9244-4455-8c07-6a46ec023d66 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:01.292070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2ce5b-9244-4455-8c07-6a46ec023d66 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:01.315205 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:01.326687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39f329-2292-4d15-844d-d4903908f11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:01.328757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39f329-2292-4d15-844d-d4903908f11b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:01.328757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39f329-2292-4d15-844d-d4903908f11b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:01.328757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39f329-2292-4d15-844d-d4903908f11b None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:01.328757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39f329-2292-4d15-844d-d4903908f11b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:01.328757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39f329-2292-4d15-844d-d4903908f11b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:01.330234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39f329-2292-4d15-844d-d4903908f11b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:01.342495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 544/1090] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:01.348278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b401c-7518-4395-ad68-6b9eadb3b0d6 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:01.407972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:01.455713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b401c-7518-4395-ad68-6b9eadb3b0d6 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:01.488934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2e97c-c7f6-4a43-a3ae-08d2f227f38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:01.489562 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2e97c-c7f6-4a43-a3ae-08d2f227f38e None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:01.489843 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2e97c-c7f6-4a43-a3ae-08d2f227f38e None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:01.490120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2e97c-c7f6-4a43-a3ae-08d2f227f38e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:01.498008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2e97c-c7f6-4a43-a3ae-08d2f227f38e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:01.499437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2e97c-c7f6-4a43-a3ae-08d2f227f38e None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-afa2e97c-c7f6-4a43-a3ae-08d2f227f38e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:07:01.510746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:01.511682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:01.512690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:01.512690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:01.512690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:07:01.512690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:07:01.512690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:07:01.512690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:07:01.512690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:07:01.512690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:07:01.512690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:07:01.512690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:07:01.512690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 547/1091] 10.222.0.85 () {56 vars in 1093 bytes} [Mon Aug 29 20:07:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:07:01.518322 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39f329-2292-4d15-844d-d4903908f11b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:01.621766 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39f329-2292-4d15-844d-d4903908f11b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:01.655867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f1130-f0ca-45d0-b932-37c453499972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:01.656502 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f1130-f0ca-45d0-b932-37c453499972 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:01.656781 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f1130-f0ca-45d0-b932-37c453499972 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:01.657052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f1130-f0ca-45d0-b932-37c453499972 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:01.657772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f1130-f0ca-45d0-b932-37c453499972 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:01.658423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f1130-f0ca-45d0-b932-37c453499972 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:01.659902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f1130-f0ca-45d0-b932-37c453499972 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:01.676497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 545/1092] 10.222.0.85 () {56 vars in 1133 bytes} [Mon Aug 29 20:07:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:01.682183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2e97c-c7f6-4a43-a3ae-08d2f227f38e None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:01.779760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2e97c-c7f6-4a43-a3ae-08d2f227f38e None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:01.796352 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:01.813173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91502a3a-fa97-416f-a12b-f2a474e9202c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:01.813774 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91502a3a-fa97-416f-a12b-f2a474e9202c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:01.814711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91502a3a-fa97-416f-a12b-f2a474e9202c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:01.814711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91502a3a-fa97-416f-a12b-f2a474e9202c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:01.880389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91502a3a-fa97-416f-a12b-f2a474e9202c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:01.881837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91502a3a-fa97-416f-a12b-f2a474e9202c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:01.986530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 548/1093] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:07:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:07:01.988053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f1130-f0ca-45d0-b932-37c453499972 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:02.049319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:02.105462 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f1130-f0ca-45d0-b932-37c453499972 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:02.138844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7675f-b531-493f-a5b3-6d15044b92b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:02.139467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7675f-b531-493f-a5b3-6d15044b92b9 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:02.139743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7675f-b531-493f-a5b3-6d15044b92b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:02.140014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7675f-b531-493f-a5b3-6d15044b92b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:02.150399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c7675f-b531-493f-a5b3-6d15044b92b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:02.151837 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c7675f-b531-493f-a5b3-6d15044b92b9 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-85c7675f-b531-493f-a5b3-6d15044b92b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:07:02.166314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:02.167298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:07:02.168756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:07:02.168756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:07:02.168756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:07:02.168756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 546/1094] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:07:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:07:02.173618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91502a3a-fa97-416f-a12b-f2a474e9202c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:02.191731 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:02.270123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91502a3a-fa97-416f-a12b-f2a474e9202c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:02.303720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53489c1b-7a65-4cb1-a47c-4388ac654e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:02.304353 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53489c1b-7a65-4cb1-a47c-4388ac654e3a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:02.304632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53489c1b-7a65-4cb1-a47c-4388ac654e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:02.304903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53489c1b-7a65-4cb1-a47c-4388ac654e3a None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:02.305541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53489c1b-7a65-4cb1-a47c-4388ac654e3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:02.305844 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53489c1b-7a65-4cb1-a47c-4388ac654e3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:02.307203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53489c1b-7a65-4cb1-a47c-4388ac654e3a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:02.318755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 549/1095] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:02.324410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7675f-b531-493f-a5b3-6d15044b92b9 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:02.415230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:02.423859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7675f-b531-493f-a5b3-6d15044b92b9 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:02.458860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cac7a-b8a1-4fa9-9171-8ac57979f78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:02.459482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cac7a-b8a1-4fa9-9171-8ac57979f78c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:02.459755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cac7a-b8a1-4fa9-9171-8ac57979f78c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:02.460059 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cac7a-b8a1-4fa9-9171-8ac57979f78c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:02.469918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cac7a-b8a1-4fa9-9171-8ac57979f78c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:02.472820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cac7a-b8a1-4fa9-9171-8ac57979f78c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-aa8cac7a-b8a1-4fa9-9171-8ac57979f78c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:07:02.484064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:02.485026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:02.486560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:07:02.486560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:07:02.486560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:07:02.486560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:07:02.486560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:07:02.486560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:07:02.486560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:07:02.486560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:07:02.486560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 547/1096] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:07:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:07:02.491576 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53489c1b-7a65-4cb1-a47c-4388ac654e3a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:02.570871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:02.663652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53489c1b-7a65-4cb1-a47c-4388ac654e3a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:02.663652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96cda5-7a09-475d-b5a8-fceaf254c17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:02.663652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96cda5-7a09-475d-b5a8-fceaf254c17d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:02.663652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96cda5-7a09-475d-b5a8-fceaf254c17d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:02.664518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96cda5-7a09-475d-b5a8-fceaf254c17d None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:02.664518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96cda5-7a09-475d-b5a8-fceaf254c17d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:02.664518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96cda5-7a09-475d-b5a8-fceaf254c17d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:02.664518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96cda5-7a09-475d-b5a8-fceaf254c17d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:02.665631 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 550/1097] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:07:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:02.671178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cac7a-b8a1-4fa9-9171-8ac57979f78c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:02.761100 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a7618-cb8e-412b-8513-58d6ebd5689e None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:02.761407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a7618-cb8e-412b-8513-58d6ebd5689e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:02.761639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a7618-cb8e-412b-8513-58d6ebd5689e None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:02.763494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 551/1098] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:07:02.768623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24455cf3-98a9-4505-a29c-7bf076633429 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:02.768903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24455cf3-98a9-4505-a29c-7bf076633429 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:02.769172 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24455cf3-98a9-4505-a29c-7bf076633429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:02.777454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cac7a-b8a1-4fa9-9171-8ac57979f78c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:02.787594 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-24455cf3-98a9-4505-a29c-7bf076633429 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:02.788828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-24455cf3-98a9-4505-a29c-7bf076633429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:02.810984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce84fef-25f6-4f6f-b75e-d525b5917142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:02.811582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce84fef-25f6-4f6f-b75e-d525b5917142 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:02.811862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce84fef-25f6-4f6f-b75e-d525b5917142 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:02.812380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce84fef-25f6-4f6f-b75e-d525b5917142 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:02.812888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce84fef-25f6-4f6f-b75e-d525b5917142 None admin] RBAC: Adding query filter params (role.id=130698a8e0c6468491e7d81752709939, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=805df3047fee45c7a702d72a5c082eb5) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:02.813199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce84fef-25f6-4f6f-b75e-d525b5917142 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:02.814616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce84fef-25f6-4f6f-b75e-d525b5917142 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:02.820443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 548/1099] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:07:02 2022] GET /identity/v3/role_assignments?user.id=805df3047fee45c7a702d72a5c082eb5&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=130698a8e0c6468491e7d81752709939 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:02.858228 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 552/1100] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:07:03.136331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce84fef-25f6-4f6f-b75e-d525b5917142 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:03.156076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:03.241131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce84fef-25f6-4f6f-b75e-d525b5917142 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:03.276232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a12959-7f71-4343-baa9-6158083ac12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:03.276876 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a12959-7f71-4343-baa9-6158083ac12c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:03.277154 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a12959-7f71-4343-baa9-6158083ac12c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:03.277429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a12959-7f71-4343-baa9-6158083ac12c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:03.290736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a12959-7f71-4343-baa9-6158083ac12c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-33a12959-7f71-4343-baa9-6158083ac12c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:07:03.295663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:03.296933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:03.298198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:03.298198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:03.298198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:03.298198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:03.298198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:07:03.298198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:07:03.298198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:07:03.298198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:07:03.298198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:07:03.298198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:07:03.298198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:03.298198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:07:03.298198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 549/1101] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:07:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:07:03.302799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24455cf3-98a9-4505-a29c-7bf076633429 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:03.370320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:03.404531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24455cf3-98a9-4505-a29c-7bf076633429 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:03.441259 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e905ff5-2021-44cf-bd86-89a2f7a0932c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:03.441874 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e905ff5-2021-44cf-bd86-89a2f7a0932c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:03.442156 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e905ff5-2021-44cf-bd86-89a2f7a0932c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:03.442434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e905ff5-2021-44cf-bd86-89a2f7a0932c None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:03.443098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e905ff5-2021-44cf-bd86-89a2f7a0932c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:03.443405 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e905ff5-2021-44cf-bd86-89a2f7a0932c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:03.444816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e905ff5-2021-44cf-bd86-89a2f7a0932c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:03.453472 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 553/1102] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:07:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:03.458918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a12959-7f71-4343-baa9-6158083ac12c None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:03.542376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:03.566595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a12959-7f71-4343-baa9-6158083ac12c None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:03.601921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7c6cc-d834-4b24-9308-142662c77a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:03.602563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7c6cc-d834-4b24-9308-142662c77a02 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:03.602841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7c6cc-d834-4b24-9308-142662c77a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:03.603113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7c6cc-d834-4b24-9308-142662c77a02 None admin] PATH_INFO: `/v3/users/859784a24733408f8aac3ad9537602e9` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:03.604299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7c6cc-d834-4b24-9308-142662c77a02 None admin] RBAC: Authorizing `identity:get_user(user_id=859784a24733408f8aac3ad9537602e9)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:03.605755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7c6cc-d834-4b24-9308-142662c77a02 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:03.608067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 550/1103] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:03 2022] GET /identity/v3/users/859784a24733408f8aac3ad9537602e9 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:03.614071 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e905ff5-2021-44cf-bd86-89a2f7a0932c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:03.714810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e905ff5-2021-44cf-bd86-89a2f7a0932c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:03.742137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:03.748249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24304317-2144-430d-b673-2e3ec777fc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:03.748847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24304317-2144-430d-b673-2e3ec777fc61 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:03.749121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24304317-2144-430d-b673-2e3ec777fc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:03.749386 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24304317-2144-430d-b673-2e3ec777fc61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:03.813500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24304317-2144-430d-b673-2e3ec777fc61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:03.814936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24304317-2144-430d-b673-2e3ec777fc61 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:03.907695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 554/1104] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:07:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:07:03.913627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7c6cc-d834-4b24-9308-142662c77a02 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:04.017964 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7c6cc-d834-4b24-9308-142662c77a02 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:04.051242 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07509528-0a82-460c-8bd0-782baa65c109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:04.051859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07509528-0a82-460c-8bd0-782baa65c109 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:04.052139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07509528-0a82-460c-8bd0-782baa65c109 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:04.052441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07509528-0a82-460c-8bd0-782baa65c109 None admin] PATH_INFO: `/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:04.053570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07509528-0a82-460c-8bd0-782baa65c109 None admin] RBAC: Authorizing `identity:get_project(project_id=efc7b01f484943d3bc4eac2e07a81d4d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:04.054978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07509528-0a82-460c-8bd0-782baa65c109 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:04.056636 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 551/1105] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:07:03 2022] GET /identity/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:04.062869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24304317-2144-430d-b673-2e3ec777fc61 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:04.092368 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:04.159676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24304317-2144-430d-b673-2e3ec777fc61 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:04.192556 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabd41a-9c4e-46d2-baa6-81aa1c2825d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:04.193156 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabd41a-9c4e-46d2-baa6-81aa1c2825d4 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:04.193435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabd41a-9c4e-46d2-baa6-81aa1c2825d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:04.193707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabd41a-9c4e-46d2-baa6-81aa1c2825d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:04.194462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabd41a-9c4e-46d2-baa6-81aa1c2825d4 None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=efc7b01f484943d3bc4eac2e07a81d4d, user.id=859784a24733408f8aac3ad9537602e9) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:04.194763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabd41a-9c4e-46d2-baa6-81aa1c2825d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:04.196203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabd41a-9c4e-46d2-baa6-81aa1c2825d4 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:04.202781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 555/1106] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:07:04 2022] GET /identity/v3/role_assignments?user.id=859784a24733408f8aac3ad9537602e9&scope.project.id=efc7b01f484943d3bc4eac2e07a81d4d&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:06.121359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700ffd5-16e4-4547-a8bf-37affafa08e7 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:06.121926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700ffd5-16e4-4547-a8bf-37affafa08e7 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:06.121926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700ffd5-16e4-4547-a8bf-37affafa08e7 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:06.123731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 552/1107] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:06.128984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe2e9a-abc1-4537-8272-053d09e01ced None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:06.129397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe2e9a-abc1-4537-8272-053d09e01ced None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:06.129536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe2e9a-abc1-4537-8272-053d09e01ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:06.147244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-85fe2e9a-abc1-4537-8272-053d09e01ced None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:06.148450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-85fe2e9a-abc1-4537-8272-053d09e01ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:06.212830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 556/1108] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:06.479161 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700ffd5-16e4-4547-a8bf-37affafa08e7 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:06.494796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:06.584194 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700ffd5-16e4-4547-a8bf-37affafa08e7 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:06.618318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e38551-d5df-4d7b-9674-2511cc3adeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:06.618946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e38551-d5df-4d7b-9674-2511cc3adeec None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:06.619222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e38551-d5df-4d7b-9674-2511cc3adeec None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:06.619880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e38551-d5df-4d7b-9674-2511cc3adeec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:06.620918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e38551-d5df-4d7b-9674-2511cc3adeec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:06.622916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e38551-d5df-4d7b-9674-2511cc3adeec None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:06.624529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 553/1109] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:07:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:06.630195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe2e9a-abc1-4537-8272-053d09e01ced None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:06.654608 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:06.726977 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe2e9a-abc1-4537-8272-053d09e01ced None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:06.760437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1bf3b-d229-45b8-a967-59625a26f25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:06.760946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1bf3b-d229-45b8-a967-59625a26f25a None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:06.761223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1bf3b-d229-45b8-a967-59625a26f25a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:06.761496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1bf3b-d229-45b8-a967-59625a26f25a None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:06.762135 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1bf3b-d229-45b8-a967-59625a26f25a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:06.763599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1bf3b-d229-45b8-a967-59625a26f25a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:06.870044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 557/1110] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:07:06 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:07:08.798679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bf3ac-737c-449a-9e5f-c060d71e1e81 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:08.799240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bf3ac-737c-449a-9e5f-c060d71e1e81 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:08.799240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bf3ac-737c-449a-9e5f-c060d71e1e81 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:08.800581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 554/1111] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:08.805966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00828d2d-5f61-441b-82df-42f34a9bd771 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:08.806357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00828d2d-5f61-441b-82df-42f34a9bd771 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:08.806523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00828d2d-5f61-441b-82df-42f34a9bd771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:08.825001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-00828d2d-5f61-441b-82df-42f34a9bd771 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:08.826224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-00828d2d-5f61-441b-82df-42f34a9bd771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:08.891022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 558/1112] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:09.158310 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bf3ac-737c-449a-9e5f-c060d71e1e81 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:09.244849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:09.265529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bf3ac-737c-449a-9e5f-c060d71e1e81 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:09.299802 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86faa5d-e0c6-442c-9443-c5c8458b6249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:09.300450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86faa5d-e0c6-442c-9443-c5c8458b6249 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:09.300731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86faa5d-e0c6-442c-9443-c5c8458b6249 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:09.301000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86faa5d-e0c6-442c-9443-c5c8458b6249 None admin] PATH_INFO: `/v3/roles/f1d8af7d158e4e02a6e77492178db259` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:09.302367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86faa5d-e0c6-442c-9443-c5c8458b6249 None admin] RBAC: Authorizing `identity:get_role(role_id=f1d8af7d158e4e02a6e77492178db259)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:09.303723 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86faa5d-e0c6-442c-9443-c5c8458b6249 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:09.305064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 555/1113] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:09 2022] GET /identity/v3/roles/f1d8af7d158e4e02a6e77492178db259 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:09.310937 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00828d2d-5f61-441b-82df-42f34a9bd771 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:09.322153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:09.411005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00828d2d-5f61-441b-82df-42f34a9bd771 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:09.445089 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2d2ae-afff-48fe-9d93-43b42db37fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:09.445713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2d2ae-afff-48fe-9d93-43b42db37fed None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:09.445992 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2d2ae-afff-48fe-9d93-43b42db37fed None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:09.446264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2d2ae-afff-48fe-9d93-43b42db37fed None admin] PATH_INFO: `/v3/users/f73e5cf5919449b8a7cecea435b7be55` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:09.460104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2d2ae-afff-48fe-9d93-43b42db37fed None admin] RBAC: Authorizing `identity:get_user(user_id=f73e5cf5919449b8a7cecea435b7be55)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:09.461568 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2d2ae-afff-48fe-9d93-43b42db37fed None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:09.463227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 559/1114] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:09 2022] GET /identity/v3/users/f73e5cf5919449b8a7cecea435b7be55 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:09.469043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86faa5d-e0c6-442c-9443-c5c8458b6249 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:09.566170 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86faa5d-e0c6-442c-9443-c5c8458b6249 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:09.599788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbafe20-7790-4557-a43e-c22daf68e69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:09.600413 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbafe20-7790-4557-a43e-c22daf68e69d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:09.600698 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbafe20-7790-4557-a43e-c22daf68e69d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:09.600974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbafe20-7790-4557-a43e-c22daf68e69d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:09.619681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:09.672102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbafe20-7790-4557-a43e-c22daf68e69d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:09.672102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbafe20-7790-4557-a43e-c22daf68e69d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:09.772825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 556/1115] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:07:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:07:09.779101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2d2ae-afff-48fe-9d93-43b42db37fed None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:09.842717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:09.874467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2d2ae-afff-48fe-9d93-43b42db37fed None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:09.908452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b52bd6-3292-4cbf-a0dd-d6e84d0c8f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:09.909071 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b52bd6-3292-4cbf-a0dd-d6e84d0c8f7a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:09.909350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b52bd6-3292-4cbf-a0dd-d6e84d0c8f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:09.909622 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b52bd6-3292-4cbf-a0dd-d6e84d0c8f7a None admin] PATH_INFO: `/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:09.910727 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b52bd6-3292-4cbf-a0dd-d6e84d0c8f7a None admin] RBAC: Authorizing `identity:get_project(project_id=efc7b01f484943d3bc4eac2e07a81d4d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:09.912140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b52bd6-3292-4cbf-a0dd-d6e84d0c8f7a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:09.913791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 560/1116] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:07:09 2022] GET /identity/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:07:09.920662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbafe20-7790-4557-a43e-c22daf68e69d None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:09.967099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:10.018084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbafe20-7790-4557-a43e-c22daf68e69d None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:10.050318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c6c78-6db2-42a0-b65c-4f2baff96cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:10.050914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c6c78-6db2-42a0-b65c-4f2baff96cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:10.051192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c6c78-6db2-42a0-b65c-4f2baff96cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:10.051463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c6c78-6db2-42a0-b65c-4f2baff96cc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:10.052184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c6c78-6db2-42a0-b65c-4f2baff96cc5 None admin] RBAC: Adding query filter params (role.id=f1d8af7d158e4e02a6e77492178db259, scope.project.id=efc7b01f484943d3bc4eac2e07a81d4d, user.id=f73e5cf5919449b8a7cecea435b7be55) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:10.052507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c6c78-6db2-42a0-b65c-4f2baff96cc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:10.053948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c6c78-6db2-42a0-b65c-4f2baff96cc5 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:10.059204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 557/1117] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:07:09 2022] GET /identity/v3/role_assignments?user.id=f73e5cf5919449b8a7cecea435b7be55&scope.project.id=efc7b01f484943d3bc4eac2e07a81d4d&role.id=f1d8af7d158e4e02a6e77492178db259 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:11.931018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b6f36-2985-4e2f-b6cc-3c78befd5eb6 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:11.931593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b6f36-2985-4e2f-b6cc-3c78befd5eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:11.931593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b6f36-2985-4e2f-b6cc-3c78befd5eb6 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:11.932928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 561/1118] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:11.938040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab45c35-6cc5-4579-8580-06c3f73e6356 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:11.938488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab45c35-6cc5-4579-8580-06c3f73e6356 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:11.938602 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab45c35-6cc5-4579-8580-06c3f73e6356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:11.956960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-cab45c35-6cc5-4579-8580-06c3f73e6356 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:11.958164 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cab45c35-6cc5-4579-8580-06c3f73e6356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:12.024103 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 558/1119] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:12.293504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b6f36-2985-4e2f-b6cc-3c78befd5eb6 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:12.392452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b6f36-2985-4e2f-b6cc-3c78befd5eb6 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:12.425698 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033ef1a-dc03-4e52-b858-43701e9532a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:12.426318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033ef1a-dc03-4e52-b858-43701e9532a1 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:12.426596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033ef1a-dc03-4e52-b858-43701e9532a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:12.426872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033ef1a-dc03-4e52-b858-43701e9532a1 None admin] PATH_INFO: `/v3/roles/f1d8af7d158e4e02a6e77492178db259` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:12.428280 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033ef1a-dc03-4e52-b858-43701e9532a1 None admin] RBAC: Authorizing `identity:get_role(role_id=f1d8af7d158e4e02a6e77492178db259)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:12.429707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033ef1a-dc03-4e52-b858-43701e9532a1 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:12.431514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 562/1120] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:12 2022] GET /identity/v3/roles/f1d8af7d158e4e02a6e77492178db259 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:12.438021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab45c35-6cc5-4579-8580-06c3f73e6356 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:12.478343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:12.533090 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab45c35-6cc5-4579-8580-06c3f73e6356 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:12.566749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb55a24-a88e-4888-bc38-8ca8fe559b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:12.567316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb55a24-a88e-4888-bc38-8ca8fe559b97 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:12.567578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb55a24-a88e-4888-bc38-8ca8fe559b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:12.567846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb55a24-a88e-4888-bc38-8ca8fe559b97 None admin] PATH_INFO: `/v3/users/f73e5cf5919449b8a7cecea435b7be55` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:12.568976 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb55a24-a88e-4888-bc38-8ca8fe559b97 None admin] RBAC: Authorizing `identity:get_user(user_id=f73e5cf5919449b8a7cecea435b7be55)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:12.570429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb55a24-a88e-4888-bc38-8ca8fe559b97 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:12.571989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 559/1121] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:12 2022] GET /identity/v3/users/f73e5cf5919449b8a7cecea435b7be55 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:12.577833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033ef1a-dc03-4e52-b858-43701e9532a1 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:12.581555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:12.676306 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033ef1a-dc03-4e52-b858-43701e9532a1 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:12.708714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7162504-e7b8-4321-b1a1-7b5e8a3a492b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:12.709342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7162504-e7b8-4321-b1a1-7b5e8a3a492b None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:12.709623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7162504-e7b8-4321-b1a1-7b5e8a3a492b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:12.709897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7162504-e7b8-4321-b1a1-7b5e8a3a492b None admin] PATH_INFO: `/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:12.710999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7162504-e7b8-4321-b1a1-7b5e8a3a492b None admin] RBAC: Authorizing `identity:get_project(project_id=efc7b01f484943d3bc4eac2e07a81d4d)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:12.712460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7162504-e7b8-4321-b1a1-7b5e8a3a492b None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:12.714096 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 563/1122] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:07:12 2022] GET /identity/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:12.720825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb55a24-a88e-4888-bc38-8ca8fe559b97 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:12.818287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb55a24-a88e-4888-bc38-8ca8fe559b97 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:12.829014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:12.852335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485dc1b1-73f3-4f12-afc7-5275beeaebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:12.853022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485dc1b1-73f3-4f12-afc7-5275beeaebd5 None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:12.853305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485dc1b1-73f3-4f12-afc7-5275beeaebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:12.853578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485dc1b1-73f3-4f12-afc7-5275beeaebd5 None admin] PATH_INFO: `/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d/users/f73e5cf5919449b8a7cecea435b7be55/roles/f1d8af7d158e4e02a6e77492178db259` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:12.856017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485dc1b1-73f3-4f12-afc7-5275beeaebd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=efc7b01f484943d3bc4eac2e07a81d4d, user_id=f73e5cf5919449b8a7cecea435b7be55, role_id=f1d8af7d158e4e02a6e77492178db259)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:12.858037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485dc1b1-73f3-4f12-afc7-5275beeaebd5 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:12.983113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 560/1123] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:07:12 2022] PUT /identity/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d/users/f73e5cf5919449b8a7cecea435b7be55/roles/f1d8af7d158e4e02a6e77492178db259 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:07:14.867965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110f9b3-757e-48c8-afcf-08fb533458f8 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:14.868508 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110f9b3-757e-48c8-afcf-08fb533458f8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:14.868508 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110f9b3-757e-48c8-afcf-08fb533458f8 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:14.869791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 564/1124] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:14.875100 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89813953-dc2a-4545-b23e-1b8604f55fc2 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:14.875531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89813953-dc2a-4545-b23e-1b8604f55fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:14.875640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89813953-dc2a-4545-b23e-1b8604f55fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:14.893928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-89813953-dc2a-4545-b23e-1b8604f55fc2 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:14.895108 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-89813953-dc2a-4545-b23e-1b8604f55fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:14.960974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 561/1125] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:15.231797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110f9b3-757e-48c8-afcf-08fb533458f8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:15.299920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:15.329441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110f9b3-757e-48c8-afcf-08fb533458f8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:15.362217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a859a03-32cf-4b6c-8f3c-08a537fa5218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:15.362832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a859a03-32cf-4b6c-8f3c-08a537fa5218 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:15.363108 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a859a03-32cf-4b6c-8f3c-08a537fa5218 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:15.363378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a859a03-32cf-4b6c-8f3c-08a537fa5218 None admin] PATH_INFO: `/v3/roles/f1d8af7d158e4e02a6e77492178db259` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:15.364764 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a859a03-32cf-4b6c-8f3c-08a537fa5218 None admin] RBAC: Authorizing `identity:get_role(role_id=f1d8af7d158e4e02a6e77492178db259)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:15.366118 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a859a03-32cf-4b6c-8f3c-08a537fa5218 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:15.367418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 565/1126] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:15 2022] GET /identity/v3/roles/f1d8af7d158e4e02a6e77492178db259 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:15.373283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89813953-dc2a-4545-b23e-1b8604f55fc2 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:15.468466 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:15.471529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89813953-dc2a-4545-b23e-1b8604f55fc2 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:15.505924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e380d25-a34c-433f-994e-5b122d6fb4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:15.506472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e380d25-a34c-433f-994e-5b122d6fb4ab None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:15.506752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e380d25-a34c-433f-994e-5b122d6fb4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:15.507029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e380d25-a34c-433f-994e-5b122d6fb4ab None admin] PATH_INFO: `/v3/users/f73e5cf5919449b8a7cecea435b7be55` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:15.508456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e380d25-a34c-433f-994e-5b122d6fb4ab None admin] RBAC: Authorizing `identity:get_user(user_id=f73e5cf5919449b8a7cecea435b7be55)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:15.510282 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e380d25-a34c-433f-994e-5b122d6fb4ab None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:15.511852 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 562/1127] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:15 2022] GET /identity/v3/users/f73e5cf5919449b8a7cecea435b7be55 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:15.517846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a859a03-32cf-4b6c-8f3c-08a537fa5218 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:15.617637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a859a03-32cf-4b6c-8f3c-08a537fa5218 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:15.651043 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67b555-3fc6-4e59-a1bc-a8a2fa21b5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:15.651636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67b555-3fc6-4e59-a1bc-a8a2fa21b5ae None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:15.651911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67b555-3fc6-4e59-a1bc-a8a2fa21b5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:15.652196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67b555-3fc6-4e59-a1bc-a8a2fa21b5ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:15.662113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:15.716855 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67b555-3fc6-4e59-a1bc-a8a2fa21b5ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:15.718305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67b555-3fc6-4e59-a1bc-a8a2fa21b5ae None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:15.813066 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 566/1128] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:07:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:07:15.819022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e380d25-a34c-433f-994e-5b122d6fb4ab None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:15.920335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e380d25-a34c-433f-994e-5b122d6fb4ab None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:15.953581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22beb0e2-29a0-4164-84c4-08eb8ba1901d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:15.954203 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beb0e2-29a0-4164-84c4-08eb8ba1901d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:15.954481 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beb0e2-29a0-4164-84c4-08eb8ba1901d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:15.954753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beb0e2-29a0-4164-84c4-08eb8ba1901d None admin] PATH_INFO: `/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:15.955857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22beb0e2-29a0-4164-84c4-08eb8ba1901d None admin] RBAC: Authorizing `identity:get_project(project_id=efc7b01f484943d3bc4eac2e07a81d4d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:15.957308 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22beb0e2-29a0-4164-84c4-08eb8ba1901d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:15.958932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 563/1129] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:07:15 2022] GET /identity/v3/projects/efc7b01f484943d3bc4eac2e07a81d4d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:15.965117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67b555-3fc6-4e59-a1bc-a8a2fa21b5ae None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:16.005836 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:16.064849 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67b555-3fc6-4e59-a1bc-a8a2fa21b5ae None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:16.098109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c09b88-b6c1-4b29-ad8b-ec5aadbe8df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:16.098891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c09b88-b6c1-4b29-ad8b-ec5aadbe8df9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:16.099375 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c09b88-b6c1-4b29-ad8b-ec5aadbe8df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:16.099860 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c09b88-b6c1-4b29-ad8b-ec5aadbe8df9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:16.101184 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c09b88-b6c1-4b29-ad8b-ec5aadbe8df9 None admin] RBAC: Adding query filter params (role.id=f1d8af7d158e4e02a6e77492178db259, scope.project.id=efc7b01f484943d3bc4eac2e07a81d4d, user.id=f73e5cf5919449b8a7cecea435b7be55) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:16.101513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c09b88-b6c1-4b29-ad8b-ec5aadbe8df9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:16.103646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c09b88-b6c1-4b29-ad8b-ec5aadbe8df9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:16.110001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 567/1130] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:07:15 2022] GET /identity/v3/role_assignments?user.id=f73e5cf5919449b8a7cecea435b7be55&scope.project.id=efc7b01f484943d3bc4eac2e07a81d4d&role.id=f1d8af7d158e4e02a6e77492178db259 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:18.021161 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51405111-1220-4af8-a7cf-47044e13e471 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:18.021717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51405111-1220-4af8-a7cf-47044e13e471 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:18.021717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51405111-1220-4af8-a7cf-47044e13e471 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:18.023022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 564/1131] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:18.029100 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b05fa-f36a-4a2f-8e01-bf2352400896 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:18.029507 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b05fa-f36a-4a2f-8e01-bf2352400896 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:18.029648 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b05fa-f36a-4a2f-8e01-bf2352400896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:18.047826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-880b05fa-f36a-4a2f-8e01-bf2352400896 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:18.049073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-880b05fa-f36a-4a2f-8e01-bf2352400896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:18.119233 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 568/1132] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:18.386864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51405111-1220-4af8-a7cf-47044e13e471 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:18.423560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:18.489555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51405111-1220-4af8-a7cf-47044e13e471 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:18.522260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55979f37-cb93-40e2-ab5e-abf58b276007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:18.522878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55979f37-cb93-40e2-ab5e-abf58b276007 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:18.523158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55979f37-cb93-40e2-ab5e-abf58b276007 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:18.523430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55979f37-cb93-40e2-ab5e-abf58b276007 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:18.524820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55979f37-cb93-40e2-ab5e-abf58b276007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:18.526249 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55979f37-cb93-40e2-ab5e-abf58b276007 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:18.528535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 565/1133] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:07:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:18.534524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b05fa-f36a-4a2f-8e01-bf2352400896 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:18.566670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:18.635396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b05fa-f36a-4a2f-8e01-bf2352400896 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:18.667805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef657e8-e7a4-4cce-b9f0-5c3daa6b62f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:18.668934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef657e8-e7a4-4cce-b9f0-5c3daa6b62f6 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:18.669218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef657e8-e7a4-4cce-b9f0-5c3daa6b62f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:18.669488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef657e8-e7a4-4cce-b9f0-5c3daa6b62f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:18.670137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef657e8-e7a4-4cce-b9f0-5c3daa6b62f6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:18.671567 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef657e8-e7a4-4cce-b9f0-5c3daa6b62f6 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:18.790484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 569/1134] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:07:18 2022] POST /identity/v3/projects => generated 311 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:07:20.719132 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768a85e-5ea5-4de3-ab23-cce9674b2284 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:20.719683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768a85e-5ea5-4de3-ab23-cce9674b2284 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:20.719683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768a85e-5ea5-4de3-ab23-cce9674b2284 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:20.721044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 566/1135] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:20.726254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b186f-30f4-4043-9b4a-8491e928101f None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:20.726642 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b186f-30f4-4043-9b4a-8491e928101f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:20.726812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b186f-30f4-4043-9b4a-8491e928101f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:20.745740 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-9a6b186f-30f4-4043-9b4a-8491e928101f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:20.747625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9a6b186f-30f4-4043-9b4a-8491e928101f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:20.816964 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 570/1136] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:21.087201 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768a85e-5ea5-4de3-ab23-cce9674b2284 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:21.188555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768a85e-5ea5-4de3-ab23-cce9674b2284 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:21.190841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:21.221968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d241d-4f46-4a4e-b298-a3d06c76363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:21.222591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d241d-4f46-4a4e-b298-a3d06c76363c None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:21.222872 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d241d-4f46-4a4e-b298-a3d06c76363c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:21.223145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d241d-4f46-4a4e-b298-a3d06c76363c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:21.224276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d241d-4f46-4a4e-b298-a3d06c76363c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:21.225702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d241d-4f46-4a4e-b298-a3d06c76363c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:21.227267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 567/1137] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:07:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:21.233806 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b186f-30f4-4043-9b4a-8491e928101f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:21.250851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:21.331886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b186f-30f4-4043-9b4a-8491e928101f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:21.364871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1925c2-4f03-45da-8db1-07cd6706d6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:21.365385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1925c2-4f03-45da-8db1-07cd6706d6b7 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:21.365661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1925c2-4f03-45da-8db1-07cd6706d6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:21.365931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1925c2-4f03-45da-8db1-07cd6706d6b7 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:21.366571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1925c2-4f03-45da-8db1-07cd6706d6b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:21.368001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1925c2-4f03-45da-8db1-07cd6706d6b7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:21.446542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 571/1138] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:07:21 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:07:23.393803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924efd3-b9a3-4c04-9b72-321853d59ee5 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:23.394380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924efd3-b9a3-4c04-9b72-321853d59ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:23.394380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924efd3-b9a3-4c04-9b72-321853d59ee5 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:23.395694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 568/1139] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:23.401044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491847-e68c-415c-bd26-313e21102651 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:23.401488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491847-e68c-415c-bd26-313e21102651 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:23.401581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491847-e68c-415c-bd26-313e21102651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:23.420550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-62491847-e68c-415c-bd26-313e21102651 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:23.421733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-62491847-e68c-415c-bd26-313e21102651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:23.485387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 572/1140] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:23.753321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924efd3-b9a3-4c04-9b72-321853d59ee5 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:23.853871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924efd3-b9a3-4c04-9b72-321853d59ee5 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:23.890595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dd4ab-b75a-4bc1-8cf4-c318316b1f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:23.891227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dd4ab-b75a-4bc1-8cf4-c318316b1f89 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:23.891509 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dd4ab-b75a-4bc1-8cf4-c318316b1f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:23.891787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dd4ab-b75a-4bc1-8cf4-c318316b1f89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:23.902870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dd4ab-b75a-4bc1-8cf4-c318316b1f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-ab4dd4ab-b75a-4bc1-8cf4-c318316b1f89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:07:23.907706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:23.908814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:23.909692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:23.909692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:23.909692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:23.909692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:23.909692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:07:23.909692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:07:23.909692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:07:23.909692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:07:23.909692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:07:23.909692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:07:23.909692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:23.909692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:07:23.909692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 569/1141] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:07:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:07:23.914883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62491847-e68c-415c-bd26-313e21102651 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:23.917081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:24.014210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62491847-e68c-415c-bd26-313e21102651 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:24.049651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f016c-fd1d-412c-8982-aaf969d7f851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:24.050251 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f016c-fd1d-412c-8982-aaf969d7f851 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:24.050530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f016c-fd1d-412c-8982-aaf969d7f851 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:24.050801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f016c-fd1d-412c-8982-aaf969d7f851 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:24.053559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f016c-fd1d-412c-8982-aaf969d7f851 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:24.053559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f016c-fd1d-412c-8982-aaf969d7f851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:24.053559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f016c-fd1d-412c-8982-aaf969d7f851 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:24.061741 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 573/1142] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:07:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:24.067616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dd4ab-b75a-4bc1-8cf4-c318316b1f89 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:24.099841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:24.174212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dd4ab-b75a-4bc1-8cf4-c318316b1f89 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:24.220523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda69b79-afe0-4b48-a27a-3059d1b765f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:24.221150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda69b79-afe0-4b48-a27a-3059d1b765f6 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:24.221426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda69b79-afe0-4b48-a27a-3059d1b765f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:24.221698 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda69b79-afe0-4b48-a27a-3059d1b765f6 None admin] PATH_INFO: `/v3/users/0dc73d41db7e4dc29db1ba57540f18f6` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:24.235973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda69b79-afe0-4b48-a27a-3059d1b765f6 None admin] RBAC: Authorizing `identity:get_user(user_id=0dc73d41db7e4dc29db1ba57540f18f6)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:24.237440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda69b79-afe0-4b48-a27a-3059d1b765f6 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:24.239023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 570/1143] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:24 2022] GET /identity/v3/users/0dc73d41db7e4dc29db1ba57540f18f6 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:24.245002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f016c-fd1d-412c-8982-aaf969d7f851 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:24.306142 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:24.343631 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f016c-fd1d-412c-8982-aaf969d7f851 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:24.376885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10faf26-6324-4a98-b039-6f9b4922321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:24.377474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10faf26-6324-4a98-b039-6f9b4922321e None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:24.377749 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10faf26-6324-4a98-b039-6f9b4922321e None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:24.378017 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10faf26-6324-4a98-b039-6f9b4922321e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:24.444262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10faf26-6324-4a98-b039-6f9b4922321e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:24.445712 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10faf26-6324-4a98-b039-6f9b4922321e None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:24.503661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:24.542173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 574/1144] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:07:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:07:24.548832 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda69b79-afe0-4b48-a27a-3059d1b765f6 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:24.639833 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:24.649826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda69b79-afe0-4b48-a27a-3059d1b765f6 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:24.685878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721414bf-7739-4379-898b-9b7b82df98b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:24.686527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721414bf-7739-4379-898b-9b7b82df98b3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:24.686803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721414bf-7739-4379-898b-9b7b82df98b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:24.687069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721414bf-7739-4379-898b-9b7b82df98b3 None admin] PATH_INFO: `/v3/projects/6c888fc751fb4734bcd1bfcdd1c6ff8b` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:24.688253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721414bf-7739-4379-898b-9b7b82df98b3 None admin] RBAC: Authorizing `identity:get_project(project_id=6c888fc751fb4734bcd1bfcdd1c6ff8b)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:24.689693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721414bf-7739-4379-898b-9b7b82df98b3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:24.691316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 571/1145] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:07:24 2022] GET /identity/v3/projects/6c888fc751fb4734bcd1bfcdd1c6ff8b => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:24.697481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10faf26-6324-4a98-b039-6f9b4922321e None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:24.795451 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10faf26-6324-4a98-b039-6f9b4922321e None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:24.829354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7f845-9cd8-43b5-9045-2dfbc87af253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:24.829945 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7f845-9cd8-43b5-9045-2dfbc87af253 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:24.830219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7f845-9cd8-43b5-9045-2dfbc87af253 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:24.830493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7f845-9cd8-43b5-9045-2dfbc87af253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:24.831233 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7f845-9cd8-43b5-9045-2dfbc87af253 None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=6c888fc751fb4734bcd1bfcdd1c6ff8b, user.id=0dc73d41db7e4dc29db1ba57540f18f6) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:24.831534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7f845-9cd8-43b5-9045-2dfbc87af253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:24.832999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7f845-9cd8-43b5-9045-2dfbc87af253 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:24.838131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 575/1146] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:07:24 2022] GET /identity/v3/role_assignments?user.id=0dc73d41db7e4dc29db1ba57540f18f6&scope.project.id=6c888fc751fb4734bcd1bfcdd1c6ff8b&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:26.728820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891982a-d487-4b1a-9cc1-c610803125e8 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:26.729352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891982a-d487-4b1a-9cc1-c610803125e8 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:26.729352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891982a-d487-4b1a-9cc1-c610803125e8 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:26.730709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 572/1147] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:26.735759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a33c4-9cbd-4b5b-ae0a-35a00397bebb None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:26.736221 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a33c4-9cbd-4b5b-ae0a-35a00397bebb None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:26.736341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a33c4-9cbd-4b5b-ae0a-35a00397bebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:26.754292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-598a33c4-9cbd-4b5b-ae0a-35a00397bebb None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:26.755469 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-598a33c4-9cbd-4b5b-ae0a-35a00397bebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:26.778736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:26.820697 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 576/1148] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:27.089290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891982a-d487-4b1a-9cc1-c610803125e8 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:27.190751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891982a-d487-4b1a-9cc1-c610803125e8 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:27.223305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667ebd5-f359-41d9-85e7-2dec554d4585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:27.223917 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667ebd5-f359-41d9-85e7-2dec554d4585 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:27.224213 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667ebd5-f359-41d9-85e7-2dec554d4585 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:27.224485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667ebd5-f359-41d9-85e7-2dec554d4585 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:27.234921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667ebd5-f359-41d9-85e7-2dec554d4585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-f667ebd5-f359-41d9-85e7-2dec554d4585 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:07:27.239683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:27.240820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:27.241863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:27.241863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:27.241863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:27.241863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:27.241863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:07:27.241863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:07:27.241863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:07:27.241863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:07:27.241863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:07:27.241863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:07:27.241863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:27.241863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:07:27.241863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 573/1149] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:07:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:07:27.246969 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a33c4-9cbd-4b5b-ae0a-35a00397bebb None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:27.342957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a33c4-9cbd-4b5b-ae0a-35a00397bebb None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:27.375966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224da4b-040d-4ced-96cc-d7c37c6f8be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:27.376593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224da4b-040d-4ced-96cc-d7c37c6f8be0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:27.376868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224da4b-040d-4ced-96cc-d7c37c6f8be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:27.377139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224da4b-040d-4ced-96cc-d7c37c6f8be0 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:27.377800 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224da4b-040d-4ced-96cc-d7c37c6f8be0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:27.378108 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224da4b-040d-4ced-96cc-d7c37c6f8be0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:27.379474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224da4b-040d-4ced-96cc-d7c37c6f8be0 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:27.388098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 577/1150] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:07:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:27.394082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667ebd5-f359-41d9-85e7-2dec554d4585 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:27.413628 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:27.492932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667ebd5-f359-41d9-85e7-2dec554d4585 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:27.526881 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d372dc6-be79-4820-bcb6-a14c6ac2ffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:27.528125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d372dc6-be79-4820-bcb6-a14c6ac2ffe4 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:27.528436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d372dc6-be79-4820-bcb6-a14c6ac2ffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:27.528703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d372dc6-be79-4820-bcb6-a14c6ac2ffe4 None admin] PATH_INFO: `/v3/users/0dc73d41db7e4dc29db1ba57540f18f6` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:27.529829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d372dc6-be79-4820-bcb6-a14c6ac2ffe4 None admin] RBAC: Authorizing `identity:get_user(user_id=0dc73d41db7e4dc29db1ba57540f18f6)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:27.531443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d372dc6-be79-4820-bcb6-a14c6ac2ffe4 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:27.533306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 574/1151] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:27 2022] GET /identity/v3/users/0dc73d41db7e4dc29db1ba57540f18f6 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:27.539153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224da4b-040d-4ced-96cc-d7c37c6f8be0 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:27.560832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:27.639466 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224da4b-040d-4ced-96cc-d7c37c6f8be0 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:27.672753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f5e16-7ceb-405f-a60b-08d93c8986a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:27.673382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f5e16-7ceb-405f-a60b-08d93c8986a8 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:27.673661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f5e16-7ceb-405f-a60b-08d93c8986a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:27.673934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f5e16-7ceb-405f-a60b-08d93c8986a8 None admin] PATH_INFO: `/v3/projects/6c888fc751fb4734bcd1bfcdd1c6ff8b` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:27.675034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f5e16-7ceb-405f-a60b-08d93c8986a8 None admin] RBAC: Authorizing `identity:get_project(project_id=6c888fc751fb4734bcd1bfcdd1c6ff8b)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:27.676579 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f5e16-7ceb-405f-a60b-08d93c8986a8 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:27.678218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 578/1152] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:07:27 2022] GET /identity/v3/projects/6c888fc751fb4734bcd1bfcdd1c6ff8b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:27.684298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d372dc6-be79-4820-bcb6-a14c6ac2ffe4 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:27.766426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:27.783200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d372dc6-be79-4820-bcb6-a14c6ac2ffe4 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:27.816608 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cda8f-4115-4407-bee9-d9521dd49627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:27.817258 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cda8f-4115-4407-bee9-d9521dd49627 None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:27.817538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cda8f-4115-4407-bee9-d9521dd49627 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:27.817814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cda8f-4115-4407-bee9-d9521dd49627 None admin] PATH_INFO: `/v3/projects/6c888fc751fb4734bcd1bfcdd1c6ff8b/users/0dc73d41db7e4dc29db1ba57540f18f6/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:27.819808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cda8f-4115-4407-bee9-d9521dd49627 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c888fc751fb4734bcd1bfcdd1c6ff8b, user_id=0dc73d41db7e4dc29db1ba57540f18f6, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:27.821256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cda8f-4115-4407-bee9-d9521dd49627 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:27.926653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 575/1153] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:07:27 2022] PUT /identity/v3/projects/6c888fc751fb4734bcd1bfcdd1c6ff8b/users/0dc73d41db7e4dc29db1ba57540f18f6/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:07:29.831932 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4b223-5ace-45c2-a6a9-de4fb450c5ee None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:29.832501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4b223-5ace-45c2-a6a9-de4fb450c5ee None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:29.832501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4b223-5ace-45c2-a6a9-de4fb450c5ee None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:29.833839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 579/1154] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:29.839078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c59e7-b1a7-4a19-a1d0-9bf47264f51a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:29.839484 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c59e7-b1a7-4a19-a1d0-9bf47264f51a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:29.839660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c59e7-b1a7-4a19-a1d0-9bf47264f51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:29.860671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-209c59e7-b1a7-4a19-a1d0-9bf47264f51a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:29.861871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-209c59e7-b1a7-4a19-a1d0-9bf47264f51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:29.928798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 576/1155] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:30.198408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4b223-5ace-45c2-a6a9-de4fb450c5ee None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:30.286212 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:30.299269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4b223-5ace-45c2-a6a9-de4fb450c5ee None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:30.332144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52f12a-3c0a-4012-9b11-83b2c4bf81f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:30.332782 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52f12a-3c0a-4012-9b11-83b2c4bf81f2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:30.333058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52f12a-3c0a-4012-9b11-83b2c4bf81f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:30.333324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52f12a-3c0a-4012-9b11-83b2c4bf81f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:30.343791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52f12a-3c0a-4012-9b11-83b2c4bf81f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-4a52f12a-3c0a-4012-9b11-83b2c4bf81f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:07:30.348660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:30.349757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:30.350680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:30.350680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:30.350680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:30.350680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:30.350680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:07:30.350680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:07:30.350680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:07:30.350680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:07:30.350680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:07:30.350680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:07:30.350680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:30.350680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:07:30.350680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 580/1156] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:07:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:07:30.356583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c59e7-b1a7-4a19-a1d0-9bf47264f51a None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:30.432138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:30.456014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c59e7-b1a7-4a19-a1d0-9bf47264f51a None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:30.489451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a3afe-1344-493f-a438-406cebafbeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:30.490042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a3afe-1344-493f-a438-406cebafbeaa None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:30.490317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a3afe-1344-493f-a438-406cebafbeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:30.490585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a3afe-1344-493f-a438-406cebafbeaa None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:30.491259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a3afe-1344-493f-a438-406cebafbeaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:30.491555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a3afe-1344-493f-a438-406cebafbeaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:30.492937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a3afe-1344-493f-a438-406cebafbeaa None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:30.500799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 577/1157] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:07:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:30.506736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52f12a-3c0a-4012-9b11-83b2c4bf81f2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:30.606119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52f12a-3c0a-4012-9b11-83b2c4bf81f2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:30.644394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268be08-3509-4448-8863-f6703b4166c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:30.645011 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268be08-3509-4448-8863-f6703b4166c7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:30.645287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268be08-3509-4448-8863-f6703b4166c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:30.645555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268be08-3509-4448-8863-f6703b4166c7 None admin] PATH_INFO: `/v3/users/0dc73d41db7e4dc29db1ba57540f18f6` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:30.646650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268be08-3509-4448-8863-f6703b4166c7 None admin] RBAC: Authorizing `identity:get_user(user_id=0dc73d41db7e4dc29db1ba57540f18f6)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:30.648065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268be08-3509-4448-8863-f6703b4166c7 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:30.649930 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 581/1158] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:30 2022] GET /identity/v3/users/0dc73d41db7e4dc29db1ba57540f18f6 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:30.656209 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a3afe-1344-493f-a438-406cebafbeaa None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:30.758362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a3afe-1344-493f-a438-406cebafbeaa None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:30.792819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506af9d-7abb-4d1a-947b-ff41e54df0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:30.793418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506af9d-7abb-4d1a-947b-ff41e54df0ef None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:30.793696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506af9d-7abb-4d1a-947b-ff41e54df0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:30.793968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506af9d-7abb-4d1a-947b-ff41e54df0ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:30.802282 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:30.858411 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506af9d-7abb-4d1a-947b-ff41e54df0ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:30.859864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506af9d-7abb-4d1a-947b-ff41e54df0ef None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:30.954280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 578/1159] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:07:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 20:07:30.960287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268be08-3509-4448-8863-f6703b4166c7 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:30.978497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:31.061510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268be08-3509-4448-8863-f6703b4166c7 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:31.095252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9b441-479c-401d-ad67-12323964166a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:31.095924 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9b441-479c-401d-ad67-12323964166a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:31.096268 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9b441-479c-401d-ad67-12323964166a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:31.096539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9b441-479c-401d-ad67-12323964166a None admin] PATH_INFO: `/v3/projects/6c888fc751fb4734bcd1bfcdd1c6ff8b` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:31.097727 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9b441-479c-401d-ad67-12323964166a None admin] RBAC: Authorizing `identity:get_project(project_id=6c888fc751fb4734bcd1bfcdd1c6ff8b)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:31.099323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9b441-479c-401d-ad67-12323964166a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:31.101150 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 582/1160] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:07:30 2022] GET /identity/v3/projects/6c888fc751fb4734bcd1bfcdd1c6ff8b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:31.107476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506af9d-7abb-4d1a-947b-ff41e54df0ef None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:31.145780 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:31.208646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506af9d-7abb-4d1a-947b-ff41e54df0ef None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:31.242012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa97ae4-9980-4f86-95b1-c407f59dc8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:31.242611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa97ae4-9980-4f86-95b1-c407f59dc8aa None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:31.242890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa97ae4-9980-4f86-95b1-c407f59dc8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:31.243160 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa97ae4-9980-4f86-95b1-c407f59dc8aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:31.243876 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa97ae4-9980-4f86-95b1-c407f59dc8aa None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=6c888fc751fb4734bcd1bfcdd1c6ff8b, user.id=0dc73d41db7e4dc29db1ba57540f18f6) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:31.244251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa97ae4-9980-4f86-95b1-c407f59dc8aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:31.245620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa97ae4-9980-4f86-95b1-c407f59dc8aa None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:31.251233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 579/1161] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:07:31 2022] GET /identity/v3/role_assignments?user.id=0dc73d41db7e4dc29db1ba57540f18f6&scope.project.id=6c888fc751fb4734bcd1bfcdd1c6ff8b&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:33.172377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d84d0a-56a8-42e6-85be-24bf763e5eb8 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:33.172913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d84d0a-56a8-42e6-85be-24bf763e5eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:33.172913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d84d0a-56a8-42e6-85be-24bf763e5eb8 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:33.174273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 583/1162] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:33.179774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207f45c-6192-4389-b83f-58fe3da2a92a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:33.180255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207f45c-6192-4389-b83f-58fe3da2a92a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:33.180352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207f45c-6192-4389-b83f-58fe3da2a92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:33.198231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-8207f45c-6192-4389-b83f-58fe3da2a92a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:33.199439 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8207f45c-6192-4389-b83f-58fe3da2a92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:33.264596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 580/1163] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:33.545188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d84d0a-56a8-42e6-85be-24bf763e5eb8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:33.632316 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:33.648552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d84d0a-56a8-42e6-85be-24bf763e5eb8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:33.682623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb9f1-e7ca-4a96-8457-4093c6a867ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:33.683237 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb9f1-e7ca-4a96-8457-4093c6a867ea None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:33.684053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb9f1-e7ca-4a96-8457-4093c6a867ea None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:33.684353 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb9f1-e7ca-4a96-8457-4093c6a867ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:33.749402 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15eb9f1-e7ca-4a96-8457-4093c6a867ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:33.750839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15eb9f1-e7ca-4a96-8457-4093c6a867ea None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:33.832462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:33.847993 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 584/1164] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:07:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:07:33.853951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207f45c-6192-4389-b83f-58fe3da2a92a None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:33.884907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:33.952697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207f45c-6192-4389-b83f-58fe3da2a92a None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:33.988023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd37da1-bb4f-4f18-8876-231f3432dcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:33.988670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd37da1-bb4f-4f18-8876-231f3432dcbe None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:33.988947 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd37da1-bb4f-4f18-8876-231f3432dcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:33.989218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd37da1-bb4f-4f18-8876-231f3432dcbe None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:33.995647 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd37da1-bb4f-4f18-8876-231f3432dcbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:33.997641 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd37da1-bb4f-4f18-8876-231f3432dcbe None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-4dd37da1-bb4f-4f18-8876-231f3432dcbe None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:34.008806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:34.010339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:07:34.011819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 581/1165] 10.222.0.85 () {56 vars in 1117 bytes} [Mon Aug 29 20:07:33 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:07:34.016146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb9f1-e7ca-4a96-8457-4093c6a867ea None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:34.113411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb9f1-e7ca-4a96-8457-4093c6a867ea None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:34.146022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fbb8d-3003-402e-aa1d-7acc34c17405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:34.146617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fbb8d-3003-402e-aa1d-7acc34c17405 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:34.146896 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fbb8d-3003-402e-aa1d-7acc34c17405 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:34.147167 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fbb8d-3003-402e-aa1d-7acc34c17405 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:34.147804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fbb8d-3003-402e-aa1d-7acc34c17405 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:34.148107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fbb8d-3003-402e-aa1d-7acc34c17405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:34.149840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fbb8d-3003-402e-aa1d-7acc34c17405 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:34.157346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 585/1166] 10.222.0.85 () {56 vars in 1115 bytes} [Mon Aug 29 20:07:34 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:36.060922 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e06c96-305d-4e90-96bc-370d6f050f66 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:36.061479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e06c96-305d-4e90-96bc-370d6f050f66 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:36.061479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e06c96-305d-4e90-96bc-370d6f050f66 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:36.062805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 582/1167] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:36.067876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ccb64-a3df-4148-b74a-1469d2110146 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:36.068288 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ccb64-a3df-4148-b74a-1469d2110146 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:36.068478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ccb64-a3df-4148-b74a-1469d2110146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:36.086251 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-e96ccb64-a3df-4148-b74a-1469d2110146 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:36.087437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e96ccb64-a3df-4148-b74a-1469d2110146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:36.130058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:36.152757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 586/1168] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:36.426237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e06c96-305d-4e90-96bc-370d6f050f66 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:36.516098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:36.525631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e06c96-305d-4e90-96bc-370d6f050f66 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:36.559662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897c78c-0f1e-4c82-853c-be5f48f6b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:36.560290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897c78c-0f1e-4c82-853c-be5f48f6b25c None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:36.560571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897c78c-0f1e-4c82-853c-be5f48f6b25c None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:36.560842 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897c78c-0f1e-4c82-853c-be5f48f6b25c None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:36.561509 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897c78c-0f1e-4c82-853c-be5f48f6b25c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:36.562871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897c78c-0f1e-4c82-853c-be5f48f6b25c None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:36.648066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 583/1169] 10.222.0.85 () {60 vars in 1138 bytes} [Mon Aug 29 20:07:36 2022] POST /identity/v3/domains => generated 267 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:07:38.566631 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6089d-c545-4b93-8095-6b0cc4710e40 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:38.567198 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6089d-c545-4b93-8095-6b0cc4710e40 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:38.567198 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6089d-c545-4b93-8095-6b0cc4710e40 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:38.568538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 587/1170] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:38.573626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709e7b7-8918-43db-a900-bb700be78e85 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:38.574138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709e7b7-8918-43db-a900-bb700be78e85 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:38.574240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709e7b7-8918-43db-a900-bb700be78e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:38.592711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-a709e7b7-8918-43db-a900-bb700be78e85 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:38.593892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a709e7b7-8918-43db-a900-bb700be78e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:38.662080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 584/1171] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:38.928752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6089d-c545-4b93-8095-6b0cc4710e40 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:39.030939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6089d-c545-4b93-8095-6b0cc4710e40 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:39.063418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df327425-37a6-44e9-bc34-58f0535fc493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:39.064054 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df327425-37a6-44e9-bc34-58f0535fc493 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:39.064358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df327425-37a6-44e9-bc34-58f0535fc493 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:39.064628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df327425-37a6-44e9-bc34-58f0535fc493 None admin] PATH_INFO: `/v3/domains/f90acede08ac4a4d9a55b3d37a19bb93` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:39.074512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df327425-37a6-44e9-bc34-58f0535fc493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f90acede08ac4a4d9a55b3d37a19bb93)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:39.075921 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df327425-37a6-44e9-bc34-58f0535fc493 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:39.077545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 588/1172] 10.222.0.85 () {56 vars in 1183 bytes} [Mon Aug 29 20:07:38 2022] GET /identity/v3/domains/f90acede08ac4a4d9a55b3d37a19bb93 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:39.083381 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709e7b7-8918-43db-a900-bb700be78e85 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:39.162693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:39.180132 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709e7b7-8918-43db-a900-bb700be78e85 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:39.214184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3433f-f9c2-4bb5-992d-5b637f33f6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:39.214820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3433f-f9c2-4bb5-992d-5b637f33f6ec None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:39.217130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3433f-f9c2-4bb5-992d-5b637f33f6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:39.217130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3433f-f9c2-4bb5-992d-5b637f33f6ec None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:39.217130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3433f-f9c2-4bb5-992d-5b637f33f6ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:39.217452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3433f-f9c2-4bb5-992d-5b637f33f6ec None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:39.305635 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 585/1173] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:07:39 2022] POST /identity/v3/projects => generated 361 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:07:41.217432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638c163-678b-471d-95af-dbff88ad5872 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:41.218048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638c163-678b-471d-95af-dbff88ad5872 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:41.218048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638c163-678b-471d-95af-dbff88ad5872 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:41.219326 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 589/1174] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:41.224895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fcfef-7421-41aa-a27f-951c25aeebce None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:41.225326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fcfef-7421-41aa-a27f-951c25aeebce None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:41.225452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fcfef-7421-41aa-a27f-951c25aeebce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:41.243453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-da5fcfef-7421-41aa-a27f-951c25aeebce None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:41.244673 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-da5fcfef-7421-41aa-a27f-951c25aeebce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:41.310840 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 586/1175] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:41.589029 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638c163-678b-471d-95af-dbff88ad5872 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:41.629285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:41.689606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638c163-678b-471d-95af-dbff88ad5872 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:41.722860 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea7233-4e9b-41b3-8ea8-4064954b9cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:41.723483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea7233-4e9b-41b3-8ea8-4064954b9cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:41.723762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea7233-4e9b-41b3-8ea8-4064954b9cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:41.724035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea7233-4e9b-41b3-8ea8-4064954b9cd6 None admin] PATH_INFO: `/v3/domains/f90acede08ac4a4d9a55b3d37a19bb93` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:41.725176 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea7233-4e9b-41b3-8ea8-4064954b9cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f90acede08ac4a4d9a55b3d37a19bb93)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:41.726609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea7233-4e9b-41b3-8ea8-4064954b9cd6 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:41.728294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 590/1176] 10.222.0.85 () {56 vars in 1183 bytes} [Mon Aug 29 20:07:41 2022] GET /identity/v3/domains/f90acede08ac4a4d9a55b3d37a19bb93 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:41.734075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fcfef-7421-41aa-a27f-951c25aeebce None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:41.795159 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:41.831449 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fcfef-7421-41aa-a27f-951c25aeebce None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:41.865114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95deb522-a95d-42a1-8d39-c1527e39be10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:41.865629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb522-a95d-42a1-8d39-c1527e39be10 None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:41.865906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb522-a95d-42a1-8d39-c1527e39be10 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:41.866173 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb522-a95d-42a1-8d39-c1527e39be10 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:41.866809 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95deb522-a95d-42a1-8d39-c1527e39be10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:41.868456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95deb522-a95d-42a1-8d39-c1527e39be10 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:42.018727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 587/1177] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:07:41 2022] POST /identity/v3/users => generated 311 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:07:43.982888 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048a434-d39a-47f8-86e2-58dc90361d8f None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:43.983469 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048a434-d39a-47f8-86e2-58dc90361d8f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:43.983572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048a434-d39a-47f8-86e2-58dc90361d8f None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:43.984918 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 591/1178] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:43.990179 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28500ca-efc2-40fc-b739-2e18a1c856a8 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:43.990624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28500ca-efc2-40fc-b739-2e18a1c856a8 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:43.990729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28500ca-efc2-40fc-b739-2e18a1c856a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:44.010044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-c28500ca-efc2-40fc-b739-2e18a1c856a8 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:44.011233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c28500ca-efc2-40fc-b739-2e18a1c856a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:44.076580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 588/1179] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:07:44.345009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048a434-d39a-47f8-86e2-58dc90361d8f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:44.445022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048a434-d39a-47f8-86e2-58dc90361d8f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:44.452531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:44.479092 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36568bb-9eed-4b30-b21b-f762f435bdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:44.480848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36568bb-9eed-4b30-b21b-f762f435bdc5 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:44.481128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36568bb-9eed-4b30-b21b-f762f435bdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:44.481397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36568bb-9eed-4b30-b21b-f762f435bdc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:44.491314 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36568bb-9eed-4b30-b21b-f762f435bdc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-d36568bb-9eed-4b30-b21b-f762f435bdc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:07:44.496717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:44.497797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:44.499048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:44.499048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:44.499048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:44.499048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:44.499048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:07:44.499048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:07:44.499048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:07:44.499048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:07:44.499048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:07:44.499048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:07:44.499048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:44.499048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:07:44.499048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 592/1180] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:07:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:07:44.503898 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28500ca-efc2-40fc-b739-2e18a1c856a8 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:44.553030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:44.600949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28500ca-efc2-40fc-b739-2e18a1c856a8 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:44.634395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480313b1-e822-42e8-8c05-e80f14f20236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:44.634992 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480313b1-e822-42e8-8c05-e80f14f20236 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:44.635272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480313b1-e822-42e8-8c05-e80f14f20236 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:44.635545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480313b1-e822-42e8-8c05-e80f14f20236 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:44.636775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480313b1-e822-42e8-8c05-e80f14f20236 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:44.637078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480313b1-e822-42e8-8c05-e80f14f20236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:44.638446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480313b1-e822-42e8-8c05-e80f14f20236 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:44.646549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 589/1181] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:07:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:44.653021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36568bb-9eed-4b30-b21b-f762f435bdc5 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:44.751571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36568bb-9eed-4b30-b21b-f762f435bdc5 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:44.784433 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77abe0-9313-4c46-b848-b4fe4587ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:44.785058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77abe0-9313-4c46-b848-b4fe4587ed24 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:44.785339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77abe0-9313-4c46-b848-b4fe4587ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:44.785615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77abe0-9313-4c46-b848-b4fe4587ed24 None admin] PATH_INFO: `/v3/users/5b78bdfd50404624b9b424409ddb7305` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:44.798800 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77abe0-9313-4c46-b848-b4fe4587ed24 None admin] RBAC: Authorizing `identity:get_user(user_id=5b78bdfd50404624b9b424409ddb7305)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:44.800269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77abe0-9313-4c46-b848-b4fe4587ed24 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:44.801867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 593/1182] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:44 2022] GET /identity/v3/users/5b78bdfd50404624b9b424409ddb7305 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:44.808463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480313b1-e822-42e8-8c05-e80f14f20236 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:44.905606 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480313b1-e822-42e8-8c05-e80f14f20236 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:44.917861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:44.938264 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c023f2-aa48-430b-85ee-d2c48597f4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:44.938864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c023f2-aa48-430b-85ee-d2c48597f4ff None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:44.939170 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c023f2-aa48-430b-85ee-d2c48597f4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:44.939436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c023f2-aa48-430b-85ee-d2c48597f4ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:45.008181 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c023f2-aa48-430b-85ee-d2c48597f4ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:45.009649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c023f2-aa48-430b-85ee-d2c48597f4ff None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:45.111780 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 590/1183] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:07:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:07:45.117678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77abe0-9313-4c46-b848-b4fe4587ed24 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:45.141933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:45.213788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77abe0-9313-4c46-b848-b4fe4587ed24 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:45.247494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f04145-4e50-41a4-a4f7-2462364820ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:45.248076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f04145-4e50-41a4-a4f7-2462364820ad None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:45.248381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f04145-4e50-41a4-a4f7-2462364820ad None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:45.248655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f04145-4e50-41a4-a4f7-2462364820ad None admin] PATH_INFO: `/v3/projects/c627fe10e4e94ff6ae3334d806db5dbf` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:45.249762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f04145-4e50-41a4-a4f7-2462364820ad None admin] RBAC: Authorizing `identity:get_project(project_id=c627fe10e4e94ff6ae3334d806db5dbf)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:45.251183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f04145-4e50-41a4-a4f7-2462364820ad None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:45.252847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 594/1184] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:07:45 2022] GET /identity/v3/projects/c627fe10e4e94ff6ae3334d806db5dbf => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:07:45.259014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c023f2-aa48-430b-85ee-d2c48597f4ff None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:45.272691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:45.361564 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c023f2-aa48-430b-85ee-d2c48597f4ff None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:45.394336 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c464238-e8c1-4833-8fa0-1d65317b4e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:45.394933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c464238-e8c1-4833-8fa0-1d65317b4e92 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:45.395214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c464238-e8c1-4833-8fa0-1d65317b4e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:45.395488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c464238-e8c1-4833-8fa0-1d65317b4e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:45.396234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c464238-e8c1-4833-8fa0-1d65317b4e92 None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=c627fe10e4e94ff6ae3334d806db5dbf, user.id=5b78bdfd50404624b9b424409ddb7305) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:45.396548 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c464238-e8c1-4833-8fa0-1d65317b4e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:45.397975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c464238-e8c1-4833-8fa0-1d65317b4e92 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:45.405118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 591/1185] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:07:45 2022] GET /identity/v3/role_assignments?user.id=5b78bdfd50404624b9b424409ddb7305&scope.project.id=c627fe10e4e94ff6ae3334d806db5dbf&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:47.311409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c280cf4-1c32-4a88-a9f4-137cc4ce531d None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:47.311972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c280cf4-1c32-4a88-a9f4-137cc4ce531d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:47.311972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c280cf4-1c32-4a88-a9f4-137cc4ce531d None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:47.313349 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 595/1186] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:47.318532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc019a-a3a6-4a00-b529-1610cd738057 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:47.318958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc019a-a3a6-4a00-b529-1610cd738057 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:47.319096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc019a-a3a6-4a00-b529-1610cd738057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:47.338567 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-45bc019a-a3a6-4a00-b529-1610cd738057 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:47.339803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-45bc019a-a3a6-4a00-b529-1610cd738057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:47.406333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 592/1187] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:47.671552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c280cf4-1c32-4a88-a9f4-137cc4ce531d None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:47.766882 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:47.776436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c280cf4-1c32-4a88-a9f4-137cc4ce531d None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:47.810198 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03fed8-f516-4c0a-a02f-20222c006ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:47.810821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03fed8-f516-4c0a-a02f-20222c006ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:47.811095 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03fed8-f516-4c0a-a02f-20222c006ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:47.811367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03fed8-f516-4c0a-a02f-20222c006ec0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:47.821111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03fed8-f516-4c0a-a02f-20222c006ec0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-7d03fed8-f516-4c0a-a02f-20222c006ec0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:07:47.826026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:47.827170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:47.828018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:47.828018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:47.828018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:47.828018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:47.828018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:07:47.828018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:07:47.828018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:07:47.828018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:07:47.828018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:07:47.828018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:07:47.828018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:47.828018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:07:47.828018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 596/1188] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:07:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:07:47.833242 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc019a-a3a6-4a00-b529-1610cd738057 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:47.841253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:47.931374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc019a-a3a6-4a00-b529-1610cd738057 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:47.966736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7595635-2e01-4abb-a920-59df30958943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:47.967344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7595635-2e01-4abb-a920-59df30958943 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:47.967627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7595635-2e01-4abb-a920-59df30958943 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:47.967897 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7595635-2e01-4abb-a920-59df30958943 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:47.968585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7595635-2e01-4abb-a920-59df30958943 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:47.968891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7595635-2e01-4abb-a920-59df30958943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:47.970255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7595635-2e01-4abb-a920-59df30958943 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:47.979045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 593/1189] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:07:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:07:47.985041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03fed8-f516-4c0a-a02f-20222c006ec0 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:48.083955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03fed8-f516-4c0a-a02f-20222c006ec0 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:48.117027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6b3ef-d0f2-4d1b-ac2e-3de2f6b96e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:48.117649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6b3ef-d0f2-4d1b-ac2e-3de2f6b96e4c None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:48.117926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6b3ef-d0f2-4d1b-ac2e-3de2f6b96e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:48.118198 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6b3ef-d0f2-4d1b-ac2e-3de2f6b96e4c None admin] PATH_INFO: `/v3/users/5b78bdfd50404624b9b424409ddb7305` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:48.119312 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6b3ef-d0f2-4d1b-ac2e-3de2f6b96e4c None admin] RBAC: Authorizing `identity:get_user(user_id=5b78bdfd50404624b9b424409ddb7305)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:48.120802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6b3ef-d0f2-4d1b-ac2e-3de2f6b96e4c None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:48.122384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 597/1190] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:47 2022] GET /identity/v3/users/5b78bdfd50404624b9b424409ddb7305 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:48.128445 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7595635-2e01-4abb-a920-59df30958943 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:48.199604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:48.233628 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7595635-2e01-4abb-a920-59df30958943 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:48.267398 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e398e-5800-4888-82b2-391cddeaeff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:48.268029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e398e-5800-4888-82b2-391cddeaeff8 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:48.268344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e398e-5800-4888-82b2-391cddeaeff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:48.268619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e398e-5800-4888-82b2-391cddeaeff8 None admin] PATH_INFO: `/v3/projects/c627fe10e4e94ff6ae3334d806db5dbf` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:48.269749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e398e-5800-4888-82b2-391cddeaeff8 None admin] RBAC: Authorizing `identity:get_project(project_id=c627fe10e4e94ff6ae3334d806db5dbf)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:48.271180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e398e-5800-4888-82b2-391cddeaeff8 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:48.272834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 594/1191] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:07:48 2022] GET /identity/v3/projects/c627fe10e4e94ff6ae3334d806db5dbf => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:48.279755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6b3ef-d0f2-4d1b-ac2e-3de2f6b96e4c None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:48.302950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:48.379970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6b3ef-d0f2-4d1b-ac2e-3de2f6b96e4c None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:48.414714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7beeb8-9902-411f-ada8-9097715d5fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:48.415367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7beeb8-9902-411f-ada8-9097715d5fe8 None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:48.415639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7beeb8-9902-411f-ada8-9097715d5fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:48.415927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7beeb8-9902-411f-ada8-9097715d5fe8 None admin] PATH_INFO: `/v3/projects/c627fe10e4e94ff6ae3334d806db5dbf/users/5b78bdfd50404624b9b424409ddb7305/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:48.418428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7beeb8-9902-411f-ada8-9097715d5fe8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c627fe10e4e94ff6ae3334d806db5dbf, user_id=5b78bdfd50404624b9b424409ddb7305, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:48.419851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7beeb8-9902-411f-ada8-9097715d5fe8 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:48.509173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 598/1192] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:07:48 2022] PUT /identity/v3/projects/c627fe10e4e94ff6ae3334d806db5dbf/users/5b78bdfd50404624b9b424409ddb7305/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:07:50.416499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6365a-bb8d-40bb-9792-b082deb7adae None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:50.417607 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6365a-bb8d-40bb-9792-b082deb7adae None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:50.417733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6365a-bb8d-40bb-9792-b082deb7adae None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:50.419082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 595/1193] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:07:50.424362 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995627eb-24ce-4cfa-84f7-cbe2c5098084 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:50.424762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995627eb-24ce-4cfa-84f7-cbe2c5098084 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:50.424915 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995627eb-24ce-4cfa-84f7-cbe2c5098084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:50.442633 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-995627eb-24ce-4cfa-84f7-cbe2c5098084 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:07:50.443809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-995627eb-24ce-4cfa-84f7-cbe2c5098084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:07:50.511149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 599/1194] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:07:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:07:50.786487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6365a-bb8d-40bb-9792-b082deb7adae None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:50.890560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6365a-bb8d-40bb-9792-b082deb7adae None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:50.923441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870eaa75-795f-49b4-a3ea-2e1e3a45b07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:50.924060 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eaa75-795f-49b4-a3ea-2e1e3a45b07f None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:50.924362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eaa75-795f-49b4-a3ea-2e1e3a45b07f None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:50.924633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eaa75-795f-49b4-a3ea-2e1e3a45b07f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:50.927181 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:50.935231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870eaa75-795f-49b4-a3ea-2e1e3a45b07f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-870eaa75-795f-49b4-a3ea-2e1e3a45b07f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:07:50.940063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:07:50.942562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:07:50.943599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:07:50.943599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:07:50.943599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:07:50.943599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:07:50.943599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:07:50.943599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:07:50.943599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:07:50.943599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:07:50.943599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:07:50.943599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:07:50.943599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:07:50.943599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:07:50.943599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 596/1195] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:07:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:07:50.947410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995627eb-24ce-4cfa-84f7-cbe2c5098084 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:50.959392 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:51.043964 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995627eb-24ce-4cfa-84f7-cbe2c5098084 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:51.079154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38aa5c-a624-4065-a62e-e89d94c5d041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:51.079757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38aa5c-a624-4065-a62e-e89d94c5d041 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:51.080041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38aa5c-a624-4065-a62e-e89d94c5d041 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:51.080349 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38aa5c-a624-4065-a62e-e89d94c5d041 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:51.081009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38aa5c-a624-4065-a62e-e89d94c5d041 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:51.081311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38aa5c-a624-4065-a62e-e89d94c5d041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:51.082678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38aa5c-a624-4065-a62e-e89d94c5d041 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:51.090828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 600/1196] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:07:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:51.096815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870eaa75-795f-49b4-a3ea-2e1e3a45b07f None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:51.193813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870eaa75-795f-49b4-a3ea-2e1e3a45b07f None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:51.225739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ba414-8e7c-414e-af7a-eabf8ac20c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:51.226353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ba414-8e7c-414e-af7a-eabf8ac20c07 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:51.226628 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ba414-8e7c-414e-af7a-eabf8ac20c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:51.226896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ba414-8e7c-414e-af7a-eabf8ac20c07 None admin] PATH_INFO: `/v3/users/5b78bdfd50404624b9b424409ddb7305` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:51.228087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ba414-8e7c-414e-af7a-eabf8ac20c07 None admin] RBAC: Authorizing `identity:get_user(user_id=5b78bdfd50404624b9b424409ddb7305)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:51.229555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ba414-8e7c-414e-af7a-eabf8ac20c07 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:51.231123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 597/1197] 10.222.0.85 () {56 vars in 1177 bytes} [Mon Aug 29 20:07:51 2022] GET /identity/v3/users/5b78bdfd50404624b9b424409ddb7305 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:51.237106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38aa5c-a624-4065-a62e-e89d94c5d041 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:51.305300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:51.334479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38aa5c-a624-4065-a62e-e89d94c5d041 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:51.367147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e7b33-4c48-486e-b49a-8e97de418409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:51.367778 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7b33-4c48-486e-b49a-8e97de418409 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:51.368089 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7b33-4c48-486e-b49a-8e97de418409 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:51.368356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7b33-4c48-486e-b49a-8e97de418409 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:51.435729 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e7b33-4c48-486e-b49a-8e97de418409 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:51.437201 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e7b33-4c48-486e-b49a-8e97de418409 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:51.503759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:51.537370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 601/1198] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:07:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:07:51.543983 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ba414-8e7c-414e-af7a-eabf8ac20c07 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:51.605234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:07:51.654186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ba414-8e7c-414e-af7a-eabf8ac20c07 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:51.687492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a892f1-28a3-4322-8d96-beef340ef694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:51.688115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a892f1-28a3-4322-8d96-beef340ef694 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:51.688424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a892f1-28a3-4322-8d96-beef340ef694 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:51.688691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a892f1-28a3-4322-8d96-beef340ef694 None admin] PATH_INFO: `/v3/projects/c627fe10e4e94ff6ae3334d806db5dbf` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:51.689783 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a892f1-28a3-4322-8d96-beef340ef694 None admin] RBAC: Authorizing `identity:get_project(project_id=c627fe10e4e94ff6ae3334d806db5dbf)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:51.691192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a892f1-28a3-4322-8d96-beef340ef694 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:51.692889 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 598/1199] 10.222.0.85 () {56 vars in 1186 bytes} [Mon Aug 29 20:07:51 2022] GET /identity/v3/projects/c627fe10e4e94ff6ae3334d806db5dbf => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:07:51.699145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e7b33-4c48-486e-b49a-8e97de418409 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:07:51.797823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e7b33-4c48-486e-b49a-8e97de418409 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:07:51.829932 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe14c3-7784-45f0-aaa6-0806a41bf765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:07:51.830523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe14c3-7784-45f0-aaa6-0806a41bf765 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:07:51.830797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe14c3-7784-45f0-aaa6-0806a41bf765 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:07:51.831062 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe14c3-7784-45f0-aaa6-0806a41bf765 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:07:51.831771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe14c3-7784-45f0-aaa6-0806a41bf765 None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=c627fe10e4e94ff6ae3334d806db5dbf, user.id=5b78bdfd50404624b9b424409ddb7305) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:07:51.832071 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe14c3-7784-45f0-aaa6-0806a41bf765 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:07:51.833520 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe14c3-7784-45f0-aaa6-0806a41bf765 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:07:51.839655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 602/1200] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:07:51 2022] GET /identity/v3/role_assignments?user.id=5b78bdfd50404624b9b424409ddb7305&scope.project.id=c627fe10e4e94ff6ae3334d806db5dbf&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:08:01.684064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=34 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:08:01.836541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=82 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:10.490323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5487eba-008d-4c0d-a91d-b129d18f9f40 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:10.490927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5487eba-008d-4c0d-a91d-b129d18f9f40 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:10.490927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5487eba-008d-4c0d-a91d-b129d18f9f40 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:10.492850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 599/1201] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:09:10.498547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc4ca6-ee4b-45c4-b7ab-827c588eae50 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:10.498969 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc4ca6-ee4b-45c4-b7ab-827c588eae50 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:10.499073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc4ca6-ee4b-45c4-b7ab-827c588eae50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:10.517735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-a2cc4ca6-ee4b-45c4-b7ab-827c588eae50 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:09:10.518922 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a2cc4ca6-ee4b-45c4-b7ab-827c588eae50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:09:10.585993 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 603/1202] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:09:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:09:10.871411 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5487eba-008d-4c0d-a91d-b129d18f9f40 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:10.976714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5487eba-008d-4c0d-a91d-b129d18f9f40 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:11.010414 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6be658-e6a0-40c2-afb0-6385206c1801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:11.021778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6be658-e6a0-40c2-afb0-6385206c1801 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:11.021778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6be658-e6a0-40c2-afb0-6385206c1801 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:11.021778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6be658-e6a0-40c2-afb0-6385206c1801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:11.023483 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6be658-e6a0-40c2-afb0-6385206c1801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:11.026458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6be658-e6a0-40c2-afb0-6385206c1801 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-2f6be658-e6a0-40c2-afb0-6385206c1801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:09:11.049803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:11.051131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:09:11.058185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:09:11.058185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:11.058185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:11.058185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 600/1203] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:11.058579 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc4ca6-ee4b-45c4-b7ab-827c588eae50 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:11.155002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc4ca6-ee4b-45c4-b7ab-827c588eae50 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:11.183726 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:11.189232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dabb63-a9aa-4fbe-8385-ed8be61d87c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:11.189865 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dabb63-a9aa-4fbe-8385-ed8be61d87c3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:11.190166 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dabb63-a9aa-4fbe-8385-ed8be61d87c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:11.190425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dabb63-a9aa-4fbe-8385-ed8be61d87c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:11.191063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dabb63-a9aa-4fbe-8385-ed8be61d87c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:11.191367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dabb63-a9aa-4fbe-8385-ed8be61d87c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:11.192769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dabb63-a9aa-4fbe-8385-ed8be61d87c3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:11.204104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 604/1204] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:11.210043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6be658-e6a0-40c2-afb0-6385206c1801 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:11.263795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:11.314901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6be658-e6a0-40c2-afb0-6385206c1801 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:11.348426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faaae7-915a-46b5-9b43-09d28aba03d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:11.352619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faaae7-915a-46b5-9b43-09d28aba03d5 None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:11.352619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faaae7-915a-46b5-9b43-09d28aba03d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:11.352619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faaae7-915a-46b5-9b43-09d28aba03d5 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:11.352619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faaae7-915a-46b5-9b43-09d28aba03d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:11.352619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faaae7-915a-46b5-9b43-09d28aba03d5 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:11.546873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 601/1205] 10.222.0.85 () {60 vars in 1132 bytes} [Mon Aug 29 20:09:11 2022] POST /identity/v3/users => generated 251 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:09:13.483171 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc811e-94fe-49bf-9126-07006c519a4f None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:13.483738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc811e-94fe-49bf-9126-07006c519a4f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:13.483738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc811e-94fe-49bf-9126-07006c519a4f None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:13.485093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 605/1206] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:09:13.490119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe96c8-2b8c-4b72-9243-3b468e55d9e4 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:13.490552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe96c8-2b8c-4b72-9243-3b468e55d9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:13.490655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe96c8-2b8c-4b72-9243-3b468e55d9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:13.509064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-a8fe96c8-2b8c-4b72-9243-3b468e55d9e4 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:09:13.510244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a8fe96c8-2b8c-4b72-9243-3b468e55d9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:09:13.577751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 602/1207] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:09:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:09:13.843841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc811e-94fe-49bf-9126-07006c519a4f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:13.945264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc811e-94fe-49bf-9126-07006c519a4f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:13.979670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736f96c-0f8d-48b6-8682-f57ac2dc0fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:13.980335 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736f96c-0f8d-48b6-8682-f57ac2dc0fbc None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:13.980609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736f96c-0f8d-48b6-8682-f57ac2dc0fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:13.980609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736f96c-0f8d-48b6-8682-f57ac2dc0fbc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:13.991221 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9736f96c-0f8d-48b6-8682-f57ac2dc0fbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-9736f96c-0f8d-48b6-8682-f57ac2dc0fbc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:09:13.996012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:14.000699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:14.001767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:14.001767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:14.001767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:14.001767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:14.001767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:09:14.001767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:09:14.001767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:09:14.001767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:09:14.001767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:09:14.001767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:09:14.001767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:09:14.001767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:14.001767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 606/1208] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:09:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:14.003346 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe96c8-2b8c-4b72-9243-3b468e55d9e4 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:14.045285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:14.103360 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe96c8-2b8c-4b72-9243-3b468e55d9e4 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:14.136902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73351da9-381e-4f39-b757-6c24596dc516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:14.137517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73351da9-381e-4f39-b757-6c24596dc516 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:14.137794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73351da9-381e-4f39-b757-6c24596dc516 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:14.138066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73351da9-381e-4f39-b757-6c24596dc516 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:14.138733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73351da9-381e-4f39-b757-6c24596dc516 None admin] RBAC: Adding query filter params (name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:14.139031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73351da9-381e-4f39-b757-6c24596dc516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:14.140429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73351da9-381e-4f39-b757-6c24596dc516 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:14.149427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 603/1209] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:09:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:14.155402 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736f96c-0f8d-48b6-8682-f57ac2dc0fbc None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:14.206863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:14.255956 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736f96c-0f8d-48b6-8682-f57ac2dc0fbc None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:14.291609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b0986-f119-49bc-bb17-670901bc340d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:14.292259 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b0986-f119-49bc-bb17-670901bc340d None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:14.292750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b0986-f119-49bc-bb17-670901bc340d None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:14.292864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b0986-f119-49bc-bb17-670901bc340d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:14.302936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b0986-f119-49bc-bb17-670901bc340d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:14.304901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b0986-f119-49bc-bb17-670901bc340d None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-503b0986-f119-49bc-bb17-670901bc340d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:09:14.318995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:14.320708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:09:14.321599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:09:14.321599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:14.321599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:14.321599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 607/1210] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:14.326874 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73351da9-381e-4f39-b757-6c24596dc516 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:14.423952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:14.426843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73351da9-381e-4f39-b757-6c24596dc516 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:14.460302 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc96fec-3bb8-426f-b942-59b99ee04ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:14.460930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc96fec-3bb8-426f-b942-59b99ee04ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:14.461205 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc96fec-3bb8-426f-b942-59b99ee04ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:14.461477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc96fec-3bb8-426f-b942-59b99ee04ff3 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:14.462127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc96fec-3bb8-426f-b942-59b99ee04ff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:14.462429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc96fec-3bb8-426f-b942-59b99ee04ff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:14.463810 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc96fec-3bb8-426f-b942-59b99ee04ff3 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:14.476818 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 604/1211] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:14.482634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b0986-f119-49bc-bb17-670901bc340d None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:14.585760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b0986-f119-49bc-bb17-670901bc340d None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:14.587455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:14.620088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dcf019-292f-4cb8-af79-aa01dea9b51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:14.620745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcf019-292f-4cb8-af79-aa01dea9b51a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:14.621020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcf019-292f-4cb8-af79-aa01dea9b51a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:14.621286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcf019-292f-4cb8-af79-aa01dea9b51a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:14.628785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dcf019-292f-4cb8-af79-aa01dea9b51a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:14.630216 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dcf019-292f-4cb8-af79-aa01dea9b51a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-d5dcf019-292f-4cb8-af79-aa01dea9b51a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:09:14.641927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:14.647815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:14.649128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:14.649128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:14.649128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:09:14.649128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:09:14.649128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:09:14.649128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:09:14.649128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:09:14.649128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:09:14.649128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:09:14.649128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:14.649128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 608/1212] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:14.649882 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc96fec-3bb8-426f-b942-59b99ee04ff3 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:14.747726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc96fec-3bb8-426f-b942-59b99ee04ff3 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:14.782556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291cf4c-ccac-489a-ac10-0d68dc763bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:14.783153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291cf4c-ccac-489a-ac10-0d68dc763bea None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:14.783429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291cf4c-ccac-489a-ac10-0d68dc763bea None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:14.783698 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291cf4c-ccac-489a-ac10-0d68dc763bea None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:14.784447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291cf4c-ccac-489a-ac10-0d68dc763bea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:14.784753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291cf4c-ccac-489a-ac10-0d68dc763bea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:14.786162 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291cf4c-ccac-489a-ac10-0d68dc763bea None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:14.802926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 605/1213] 10.222.0.85 () {56 vars in 1143 bytes} [Mon Aug 29 20:09:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:14.809013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dcf019-292f-4cb8-af79-aa01dea9b51a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:14.914602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dcf019-292f-4cb8-af79-aa01dea9b51a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:14.947492 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20ae1d-038c-430a-8c8e-41b0b755a9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:14.948088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20ae1d-038c-430a-8c8e-41b0b755a9ea None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:14.948472 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20ae1d-038c-430a-8c8e-41b0b755a9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:14.948745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20ae1d-038c-430a-8c8e-41b0b755a9ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:14.978405 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:15.020713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20ae1d-038c-430a-8c8e-41b0b755a9ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:15.022080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20ae1d-038c-430a-8c8e-41b0b755a9ea None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:15.119306 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 609/1214] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:09:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:09:15.125156 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291cf4c-ccac-489a-ac10-0d68dc763bea None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:15.144931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:15.224532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291cf4c-ccac-489a-ac10-0d68dc763bea None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:15.259496 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb6a5e-deb9-4ee9-82dc-48904cf99745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:15.260109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb6a5e-deb9-4ee9-82dc-48904cf99745 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:15.260415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb6a5e-deb9-4ee9-82dc-48904cf99745 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:15.260687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb6a5e-deb9-4ee9-82dc-48904cf99745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:15.270926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb6a5e-deb9-4ee9-82dc-48904cf99745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:15.272357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb6a5e-deb9-4ee9-82dc-48904cf99745 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-27cb6a5e-deb9-4ee9-82dc-48904cf99745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:09:15.287072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:15.288355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:09:15.289268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:09:15.289268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:15.289268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:15.289268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 606/1215] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:15.294286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20ae1d-038c-430a-8c8e-41b0b755a9ea None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:15.355019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:15.394803 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20ae1d-038c-430a-8c8e-41b0b755a9ea None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:15.431321 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dcf17-9604-47fb-8cf2-c8dbf667eed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:15.432215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dcf17-9604-47fb-8cf2-c8dbf667eed3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:15.432635 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dcf17-9604-47fb-8cf2-c8dbf667eed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:15.433387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dcf17-9604-47fb-8cf2-c8dbf667eed3 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:15.434036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dcf17-9604-47fb-8cf2-c8dbf667eed3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:15.434323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dcf17-9604-47fb-8cf2-c8dbf667eed3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:15.435710 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dcf17-9604-47fb-8cf2-c8dbf667eed3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:15.447309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 610/1216] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:15.453153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb6a5e-deb9-4ee9-82dc-48904cf99745 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:15.524609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:15.555733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb6a5e-deb9-4ee9-82dc-48904cf99745 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:15.588568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c67201-f163-4a61-8e9d-9c2f14a0b188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:15.589191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c67201-f163-4a61-8e9d-9c2f14a0b188 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:15.589475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c67201-f163-4a61-8e9d-9c2f14a0b188 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:15.589776 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c67201-f163-4a61-8e9d-9c2f14a0b188 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:15.596322 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c67201-f163-4a61-8e9d-9c2f14a0b188 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:15.597761 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c67201-f163-4a61-8e9d-9c2f14a0b188 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-61c67201-f163-4a61-8e9d-9c2f14a0b188 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:09:15.609580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:15.610691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:15.611598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:09:15.611598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:09:15.611598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:09:15.611598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:09:15.611598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:09:15.611598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:09:15.611598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:09:15.611598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:15.611598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 607/1217] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:09:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:09:15.617462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dcf17-9604-47fb-8cf2-c8dbf667eed3 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:15.718195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dcf17-9604-47fb-8cf2-c8dbf667eed3 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:15.740287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:15.752039 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a0a3d-196b-4da4-b3e9-40018314c88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:15.752669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a0a3d-196b-4da4-b3e9-40018314c88a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:15.752944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a0a3d-196b-4da4-b3e9-40018314c88a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:15.753209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a0a3d-196b-4da4-b3e9-40018314c88a None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:15.753882 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a0a3d-196b-4da4-b3e9-40018314c88a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:15.754180 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a0a3d-196b-4da4-b3e9-40018314c88a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:15.755596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a0a3d-196b-4da4-b3e9-40018314c88a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:15.769858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 611/1218] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:09:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:15.776330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c67201-f163-4a61-8e9d-9c2f14a0b188 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:15.874637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c67201-f163-4a61-8e9d-9c2f14a0b188 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:15.904020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:15.909750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9a621-316c-4fe7-acb5-2219a6c8eff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:15.910350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9a621-316c-4fe7-acb5-2219a6c8eff0 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:15.910689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9a621-316c-4fe7-acb5-2219a6c8eff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:15.910901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9a621-316c-4fe7-acb5-2219a6c8eff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:15.911617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9a621-316c-4fe7-acb5-2219a6c8eff0 None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=5288af2dbc15438c8b209b0ba2454cf1) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:15.911921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9a621-316c-4fe7-acb5-2219a6c8eff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:15.913376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9a621-316c-4fe7-acb5-2219a6c8eff0 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:15.918793 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 608/1219] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:09:15 2022] GET /identity/v3/role_assignments?user.id=5288af2dbc15438c8b209b0ba2454cf1&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:17.862023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59c990-5885-4d53-8d02-c7bd110cd119 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:17.862549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59c990-5885-4d53-8d02-c7bd110cd119 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:17.862662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59c990-5885-4d53-8d02-c7bd110cd119 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:17.864154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 612/1220] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:09:17.869777 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412df15-3ce6-48ec-bdc4-15353ced5b96 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:17.870241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412df15-3ce6-48ec-bdc4-15353ced5b96 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:17.870350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412df15-3ce6-48ec-bdc4-15353ced5b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:17.889363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-f412df15-3ce6-48ec-bdc4-15353ced5b96 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:09:17.890562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-f412df15-3ce6-48ec-bdc4-15353ced5b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:09:17.958948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 609/1221] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:09:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:09:18.238811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59c990-5885-4d53-8d02-c7bd110cd119 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:18.340960 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59c990-5885-4d53-8d02-c7bd110cd119 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:18.374953 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea61cc-fd75-4e81-a1b7-e7b8449c8d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:18.375580 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea61cc-fd75-4e81-a1b7-e7b8449c8d6e None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:18.375856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea61cc-fd75-4e81-a1b7-e7b8449c8d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:18.376127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea61cc-fd75-4e81-a1b7-e7b8449c8d6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:18.385979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea61cc-fd75-4e81-a1b7-e7b8449c8d6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-daea61cc-fd75-4e81-a1b7-e7b8449c8d6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:09:18.390979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:18.392081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:18.393158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:18.393158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:18.393158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:18.393158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:18.393158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:09:18.393158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:09:18.393158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:09:18.393158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:09:18.393158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:09:18.393158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:09:18.393158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:09:18.393158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:18.393158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 613/1222] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:09:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:18.398129 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412df15-3ce6-48ec-bdc4-15353ced5b96 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:18.496298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412df15-3ce6-48ec-bdc4-15353ced5b96 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:18.519618 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:18.531365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb3088-ebb5-475d-8974-b926f10a9447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:18.531971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb3088-ebb5-475d-8974-b926f10a9447 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:18.532275 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb3088-ebb5-475d-8974-b926f10a9447 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:18.532546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb3088-ebb5-475d-8974-b926f10a9447 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:18.533198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb3088-ebb5-475d-8974-b926f10a9447 None admin] RBAC: Adding query filter params (name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:18.533894 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb3088-ebb5-475d-8974-b926f10a9447 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:18.535801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb3088-ebb5-475d-8974-b926f10a9447 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:18.544062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 610/1223] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:09:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:18.550717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea61cc-fd75-4e81-a1b7-e7b8449c8d6e None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:18.593331 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:18.648427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea61cc-fd75-4e81-a1b7-e7b8449c8d6e None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:18.681775 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4faeb9-8685-4849-8b1b-b1d1914e7cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:18.682496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4faeb9-8685-4849-8b1b-b1d1914e7cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:18.682772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4faeb9-8685-4849-8b1b-b1d1914e7cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:18.683042 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4faeb9-8685-4849-8b1b-b1d1914e7cc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:18.692887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4faeb9-8685-4849-8b1b-b1d1914e7cc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:18.694435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4faeb9-8685-4849-8b1b-b1d1914e7cc8 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-bb4faeb9-8685-4849-8b1b-b1d1914e7cc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:09:18.708965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:18.710668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:09:18.711752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:09:18.711752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:18.711752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:18.711752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 614/1224] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:18.716479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb3088-ebb5-475d-8974-b926f10a9447 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:18.814677 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb3088-ebb5-475d-8974-b926f10a9447 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:18.849253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd774ab4-3734-4dd3-86be-436a3594319b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:18.849855 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd774ab4-3734-4dd3-86be-436a3594319b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:18.850137 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd774ab4-3734-4dd3-86be-436a3594319b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:18.850412 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd774ab4-3734-4dd3-86be-436a3594319b None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:18.851049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd774ab4-3734-4dd3-86be-436a3594319b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:18.851354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd774ab4-3734-4dd3-86be-436a3594319b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:18.852753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd774ab4-3734-4dd3-86be-436a3594319b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:18.864269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 611/1225] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:18.870756 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4faeb9-8685-4849-8b1b-b1d1914e7cc8 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:18.964379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:18.970903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4faeb9-8685-4849-8b1b-b1d1914e7cc8 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:19.004848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d4865-7872-488d-8a43-28827bba0c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:19.005478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d4865-7872-488d-8a43-28827bba0c1a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:19.005762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d4865-7872-488d-8a43-28827bba0c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:19.006343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d4865-7872-488d-8a43-28827bba0c1a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:19.014679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d4865-7872-488d-8a43-28827bba0c1a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:19.016207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d4865-7872-488d-8a43-28827bba0c1a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-3e1d4865-7872-488d-8a43-28827bba0c1a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:09:19.027998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:19.029193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:19.030215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:19.030215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:19.030215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:09:19.030215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:09:19.030215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:09:19.030215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:09:19.030215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:09:19.030215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:09:19.030215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:09:19.030215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:19.030215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 615/1226] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:19.034970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd774ab4-3734-4dd3-86be-436a3594319b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:19.081923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:19.134579 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd774ab4-3734-4dd3-86be-436a3594319b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:19.168573 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc6a7b-5d3e-4491-94e3-47a612d484d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:19.169172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc6a7b-5d3e-4491-94e3-47a612d484d5 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:19.169455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc6a7b-5d3e-4491-94e3-47a612d484d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:19.169729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc6a7b-5d3e-4491-94e3-47a612d484d5 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:19.170481 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc6a7b-5d3e-4491-94e3-47a612d484d5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:19.170788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc6a7b-5d3e-4491-94e3-47a612d484d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:19.172235 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc6a7b-5d3e-4491-94e3-47a612d484d5 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:19.189221 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 612/1227] 10.222.0.85 () {56 vars in 1143 bytes} [Mon Aug 29 20:09:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:19.195578 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d4865-7872-488d-8a43-28827bba0c1a None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:19.296921 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d4865-7872-488d-8a43-28827bba0c1a None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:19.330468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7269481-e47e-45ad-b5a6-c89a75da4c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:19.331092 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7269481-e47e-45ad-b5a6-c89a75da4c66 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:19.331368 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7269481-e47e-45ad-b5a6-c89a75da4c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:19.331635 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7269481-e47e-45ad-b5a6-c89a75da4c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:19.344105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7269481-e47e-45ad-b5a6-c89a75da4c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:19.345549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7269481-e47e-45ad-b5a6-c89a75da4c66 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-d7269481-e47e-45ad-b5a6-c89a75da4c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:09:19.359540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:19.363333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:09:19.364414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:09:19.364414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:19.364414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:19.364414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 616/1228] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:19.366619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc6a7b-5d3e-4491-94e3-47a612d484d5 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:19.466638 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc6a7b-5d3e-4491-94e3-47a612d484d5 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:19.468957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:19.500472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb939aee-8005-4f46-8608-25f5ab752fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:19.501077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb939aee-8005-4f46-8608-25f5ab752fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:19.501360 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb939aee-8005-4f46-8608-25f5ab752fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:19.501636 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb939aee-8005-4f46-8608-25f5ab752fb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:19.502275 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb939aee-8005-4f46-8608-25f5ab752fb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:19.502580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb939aee-8005-4f46-8608-25f5ab752fb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:19.503986 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb939aee-8005-4f46-8608-25f5ab752fb6 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:19.516853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 613/1229] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:19.522711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7269481-e47e-45ad-b5a6-c89a75da4c66 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:19.532179 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:19.620994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7269481-e47e-45ad-b5a6-c89a75da4c66 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:19.654813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb008cc-4aa3-478b-aa90-53079a317742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:19.655439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb008cc-4aa3-478b-aa90-53079a317742 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:19.655718 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb008cc-4aa3-478b-aa90-53079a317742 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:19.655988 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb008cc-4aa3-478b-aa90-53079a317742 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:19.662848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb008cc-4aa3-478b-aa90-53079a317742 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:19.664309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb008cc-4aa3-478b-aa90-53079a317742 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-6eb008cc-4aa3-478b-aa90-53079a317742 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:09:19.678379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:19.680278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:19.681200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:09:19.681200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:09:19.681200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:09:19.681200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:09:19.681200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:09:19.681200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:09:19.681200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:09:19.681200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:19.681200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 617/1230] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:09:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:19.686074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb939aee-8005-4f46-8608-25f5ab752fb6 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:19.788308 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb939aee-8005-4f46-8608-25f5ab752fb6 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:19.825197 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ac4ee-3250-4e22-8703-44dd855fbc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:19.825796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ac4ee-3250-4e22-8703-44dd855fbc65 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:19.826076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ac4ee-3250-4e22-8703-44dd855fbc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:19.826346 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ac4ee-3250-4e22-8703-44dd855fbc65 None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:19.827033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ac4ee-3250-4e22-8703-44dd855fbc65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:19.827338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ac4ee-3250-4e22-8703-44dd855fbc65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:19.828816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ac4ee-3250-4e22-8703-44dd855fbc65 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:19.840557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 614/1231] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:09:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:19.847097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb008cc-4aa3-478b-aa90-53079a317742 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:19.912500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:19.949070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb008cc-4aa3-478b-aa90-53079a317742 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:19.984506 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ea306-2b80-47e5-bb49-ba0922882f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:19.985162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ea306-2b80-47e5-bb49-ba0922882f53 None admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:19.985495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ea306-2b80-47e5-bb49-ba0922882f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:19.985770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ea306-2b80-47e5-bb49-ba0922882f53 None admin] PATH_INFO: `/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/5288af2dbc15438c8b209b0ba2454cf1/roles/cd65aca5e0074973a10adcd1c9a180dc` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:20.000623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ea306-2b80-47e5-bb49-ba0922882f53 None admin] RBAC: Authorizing `identity:create_grant(project_id=11372fe8994543b4a19773c0e0bee53e, user_id=5288af2dbc15438c8b209b0ba2454cf1, role_id=cd65aca5e0074973a10adcd1c9a180dc)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:20.002076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ea306-2b80-47e5-bb49-ba0922882f53 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:20.083266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 618/1232] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:09:19 2022] PUT /identity/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/5288af2dbc15438c8b209b0ba2454cf1/roles/cd65aca5e0074973a10adcd1c9a180dc => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:09:21.960379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebf6f2-46cc-457b-b80d-97e20bc84b26 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:21.960941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebf6f2-46cc-457b-b80d-97e20bc84b26 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:21.960941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebf6f2-46cc-457b-b80d-97e20bc84b26 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:21.962302 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 615/1233] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:09:21.967363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9374de-217f-41b9-a20a-c15abd67d2ae None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:21.970359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9374de-217f-41b9-a20a-c15abd67d2ae None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:21.970359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9374de-217f-41b9-a20a-c15abd67d2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:21.985871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-1f9374de-217f-41b9-a20a-c15abd67d2ae None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:09:21.987042 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1f9374de-217f-41b9-a20a-c15abd67d2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:09:22.052971 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 619/1234] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:09:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:09:22.320935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ebf6f2-46cc-457b-b80d-97e20bc84b26 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:22.346505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:22.421502 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ebf6f2-46cc-457b-b80d-97e20bc84b26 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:22.454753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999cb4-2f36-4edf-b496-9687091b8713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:22.455372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999cb4-2f36-4edf-b496-9687091b8713 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:22.455655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999cb4-2f36-4edf-b496-9687091b8713 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:22.455922 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999cb4-2f36-4edf-b496-9687091b8713 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:22.466257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e999cb4-2f36-4edf-b496-9687091b8713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-4e999cb4-2f36-4edf-b496-9687091b8713 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:09:22.471012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:22.472136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:22.473305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:22.473305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:22.473305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:22.473305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:22.473305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:09:22.473305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:09:22.473305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:09:22.473305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:09:22.473305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:09:22.473305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:09:22.473305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:09:22.473305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:22.473305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 616/1235] 10.222.0.85 () {56 vars in 1102 bytes} [Mon Aug 29 20:09:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:22.478688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9374de-217f-41b9-a20a-c15abd67d2ae None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:22.525860 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:22.578556 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9374de-217f-41b9-a20a-c15abd67d2ae None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:22.610627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b4629-d509-4682-b42a-b4f29590edf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:22.611290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b4629-d509-4682-b42a-b4f29590edf8 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:22.611572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b4629-d509-4682-b42a-b4f29590edf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:22.611869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b4629-d509-4682-b42a-b4f29590edf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:22.612583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b4629-d509-4682-b42a-b4f29590edf8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:22.613436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b4629-d509-4682-b42a-b4f29590edf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:22.614240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b4629-d509-4682-b42a-b4f29590edf8 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:22.622575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 620/1236] 10.222.0.85 () {56 vars in 1103 bytes} [Mon Aug 29 20:09:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:22.628275 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999cb4-2f36-4edf-b496-9687091b8713 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:22.712633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:22.728565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999cb4-2f36-4edf-b496-9687091b8713 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:22.760866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39fd67-3c10-47fa-9548-103cd900e733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:22.761493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39fd67-3c10-47fa-9548-103cd900e733 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:22.761772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39fd67-3c10-47fa-9548-103cd900e733 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:22.762043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39fd67-3c10-47fa-9548-103cd900e733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:22.773000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39fd67-3c10-47fa-9548-103cd900e733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:22.774691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39fd67-3c10-47fa-9548-103cd900e733 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-9d39fd67-3c10-47fa-9548-103cd900e733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:09:22.789109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:22.790257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:09:22.791379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:09:22.791379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:22.791379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:22.791379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 617/1237] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:22.796211 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b4629-d509-4682-b42a-b4f29590edf8 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:22.894818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b4629-d509-4682-b42a-b4f29590edf8 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:22.904373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:22.927716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbb16f-4012-4a59-aef2-ba2188516e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:22.928355 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbb16f-4012-4a59-aef2-ba2188516e12 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:22.928634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbb16f-4012-4a59-aef2-ba2188516e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:22.928900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbb16f-4012-4a59-aef2-ba2188516e12 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:22.929528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbb16f-4012-4a59-aef2-ba2188516e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:22.929826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbb16f-4012-4a59-aef2-ba2188516e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:22.931170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbb16f-4012-4a59-aef2-ba2188516e12 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:22.943240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 621/1238] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:22.950157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39fd67-3c10-47fa-9548-103cd900e733 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:23.049968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39fd67-3c10-47fa-9548-103cd900e733 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:23.083297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348f613-61f6-43c0-9219-31efff37dbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:23.083908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348f613-61f6-43c0-9219-31efff37dbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:23.084208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348f613-61f6-43c0-9219-31efff37dbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:23.084484 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348f613-61f6-43c0-9219-31efff37dbb0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:23.087261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:23.092408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348f613-61f6-43c0-9219-31efff37dbb0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:23.093844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348f613-61f6-43c0-9219-31efff37dbb0 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-d348f613-61f6-43c0-9219-31efff37dbb0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:09:23.105167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:23.110497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:23.111779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:23.111779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:23.111779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:09:23.111779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:09:23.111779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:09:23.111779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:09:23.111779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:09:23.111779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:09:23.111779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:09:23.111779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:23.111779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 618/1239] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:23.112553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbb16f-4012-4a59-aef2-ba2188516e12 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:23.212619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbb16f-4012-4a59-aef2-ba2188516e12 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:23.245836 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d84b6e-1f47-4842-bc4e-8288ba62fad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:23.246425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d84b6e-1f47-4842-bc4e-8288ba62fad2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:23.246704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d84b6e-1f47-4842-bc4e-8288ba62fad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:23.246975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d84b6e-1f47-4842-bc4e-8288ba62fad2 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:23.247699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d84b6e-1f47-4842-bc4e-8288ba62fad2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:23.248003 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d84b6e-1f47-4842-bc4e-8288ba62fad2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:23.249480 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d84b6e-1f47-4842-bc4e-8288ba62fad2 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:23.268021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 622/1240] 10.222.0.85 () {56 vars in 1143 bytes} [Mon Aug 29 20:09:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:23.274193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348f613-61f6-43c0-9219-31efff37dbb0 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:23.374028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348f613-61f6-43c0-9219-31efff37dbb0 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:23.407826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fd8b1-3432-4727-b8ee-296ba5dd0acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:23.408451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fd8b1-3432-4727-b8ee-296ba5dd0acf None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:23.408726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fd8b1-3432-4727-b8ee-296ba5dd0acf None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:23.408995 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fd8b1-3432-4727-b8ee-296ba5dd0acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:23.472763 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:23.479111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168fd8b1-3432-4727-b8ee-296ba5dd0acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:23.480601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168fd8b1-3432-4727-b8ee-296ba5dd0acf None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:23.578523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 619/1241] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:09:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:09:23.584681 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d84b6e-1f47-4842-bc4e-8288ba62fad2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:23.614669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:23.685066 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d84b6e-1f47-4842-bc4e-8288ba62fad2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:23.719329 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375091f-5d16-40c6-8fc5-37dffd93d476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:23.719963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375091f-5d16-40c6-8fc5-37dffd93d476 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:23.720277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375091f-5d16-40c6-8fc5-37dffd93d476 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:23.720552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375091f-5d16-40c6-8fc5-37dffd93d476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:23.731301 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375091f-5d16-40c6-8fc5-37dffd93d476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:23.732717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375091f-5d16-40c6-8fc5-37dffd93d476 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-d375091f-5d16-40c6-8fc5-37dffd93d476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:09:23.747595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:23.748776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:09:23.749676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:09:23.749676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:23.749676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:23.749676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 623/1242] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:23.754604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fd8b1-3432-4727-b8ee-296ba5dd0acf None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:23.853577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:23.859206 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fd8b1-3432-4727-b8ee-296ba5dd0acf None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:23.892568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595dda7-3ea2-4e75-9c32-8f4f6532eda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:23.893167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595dda7-3ea2-4e75-9c32-8f4f6532eda2 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:23.893450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595dda7-3ea2-4e75-9c32-8f4f6532eda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:23.893722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595dda7-3ea2-4e75-9c32-8f4f6532eda2 None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:23.894358 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595dda7-3ea2-4e75-9c32-8f4f6532eda2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:23.894661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595dda7-3ea2-4e75-9c32-8f4f6532eda2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:23.896109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595dda7-3ea2-4e75-9c32-8f4f6532eda2 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:23.907988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 620/1243] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:23.914035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375091f-5d16-40c6-8fc5-37dffd93d476 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:23.993161 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:24.014102 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375091f-5d16-40c6-8fc5-37dffd93d476 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:24.046126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58666b-b6cd-4835-b2af-b74d088f8862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:24.046743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58666b-b6cd-4835-b2af-b74d088f8862 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:24.047018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58666b-b6cd-4835-b2af-b74d088f8862 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:24.047286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58666b-b6cd-4835-b2af-b74d088f8862 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:24.054239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58666b-b6cd-4835-b2af-b74d088f8862 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:24.055656 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58666b-b6cd-4835-b2af-b74d088f8862 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-3e58666b-b6cd-4835-b2af-b74d088f8862 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:09:24.067182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:24.081542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:24.082665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:09:24.082665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:09:24.082665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:09:24.082665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:09:24.082665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:09:24.082665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:09:24.082665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:09:24.082665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:24.082665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 624/1244] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:09:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:24.083346 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595dda7-3ea2-4e75-9c32-8f4f6532eda2 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:24.179531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595dda7-3ea2-4e75-9c32-8f4f6532eda2 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:24.213882 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef2bd8-f027-41dc-adcc-e7835ed4b0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:24.214484 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef2bd8-f027-41dc-adcc-e7835ed4b0af None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:24.214765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef2bd8-f027-41dc-adcc-e7835ed4b0af None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:24.215037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef2bd8-f027-41dc-adcc-e7835ed4b0af None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:24.215722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef2bd8-f027-41dc-adcc-e7835ed4b0af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:24.216029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef2bd8-f027-41dc-adcc-e7835ed4b0af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:24.217478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef2bd8-f027-41dc-adcc-e7835ed4b0af None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:24.229151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 621/1245] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:09:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:24.235476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58666b-b6cd-4835-b2af-b74d088f8862 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:24.337131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58666b-b6cd-4835-b2af-b74d088f8862 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:24.371061 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a902cf-d869-4fbf-86a4-fe4b21d76f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:24.371715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a902cf-d869-4fbf-86a4-fe4b21d76f4a None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:24.371998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a902cf-d869-4fbf-86a4-fe4b21d76f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:24.372306 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a902cf-d869-4fbf-86a4-fe4b21d76f4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:24.373021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a902cf-d869-4fbf-86a4-fe4b21d76f4a None admin] RBAC: Adding query filter params (role.id=cd65aca5e0074973a10adcd1c9a180dc, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=5288af2dbc15438c8b209b0ba2454cf1) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:24.373328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a902cf-d869-4fbf-86a4-fe4b21d76f4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:24.374738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a902cf-d869-4fbf-86a4-fe4b21d76f4a None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:24.380660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 625/1246] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:09:24 2022] GET /identity/v3/role_assignments?user.id=5288af2dbc15438c8b209b0ba2454cf1&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=cd65aca5e0074973a10adcd1c9a180dc => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:26.358904 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c7b19-5144-4f57-8492-3fada0a28c97 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:26.359465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c7b19-5144-4f57-8492-3fada0a28c97 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:26.359465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c7b19-5144-4f57-8492-3fada0a28c97 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:26.360835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 622/1247] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:09:26.366606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269dcb0-b57a-4584-ac83-31a9a4805112 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:26.367047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269dcb0-b57a-4584-ac83-31a9a4805112 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:26.367140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269dcb0-b57a-4584-ac83-31a9a4805112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:26.373474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:26.386079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-9269dcb0-b57a-4584-ac83-31a9a4805112 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:09:26.387259 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9269dcb0-b57a-4584-ac83-31a9a4805112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:09:26.451796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 626/1248] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:09:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:09:26.723794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c7b19-5144-4f57-8492-3fada0a28c97 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:26.742775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:26.827670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c7b19-5144-4f57-8492-3fada0a28c97 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:26.861742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031220a-720e-4b45-950b-71abf080e711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:26.862366 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031220a-720e-4b45-950b-71abf080e711 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:26.862644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031220a-720e-4b45-950b-71abf080e711 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:26.862916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031220a-720e-4b45-950b-71abf080e711 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:26.872996 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031220a-720e-4b45-950b-71abf080e711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-3031220a-720e-4b45-950b-71abf080e711 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:09:26.877829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:26.878916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:26.879906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:26.879906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:26.879906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:26.879906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:26.879906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:09:26.879906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:09:26.879906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:09:26.879906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:09:26.879906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:09:26.879906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:09:26.879906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:09:26.879906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:26.879906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 623/1249] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:09:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:26.885759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269dcb0-b57a-4584-ac83-31a9a4805112 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:26.983747 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269dcb0-b57a-4584-ac83-31a9a4805112 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:27.013851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:27.019746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b88f0-95e7-480f-a12c-4dbdf4995426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:27.020402 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b88f0-95e7-480f-a12c-4dbdf4995426 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:27.020682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b88f0-95e7-480f-a12c-4dbdf4995426 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:27.020953 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b88f0-95e7-480f-a12c-4dbdf4995426 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:27.021637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b88f0-95e7-480f-a12c-4dbdf4995426 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:27.021985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b88f0-95e7-480f-a12c-4dbdf4995426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:27.023377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b88f0-95e7-480f-a12c-4dbdf4995426 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:27.032478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 627/1250] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:09:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:27.038324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031220a-720e-4b45-950b-71abf080e711 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:27.117460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:27.139453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031220a-720e-4b45-950b-71abf080e711 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:27.174257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315deeb-a83e-4261-b529-868ab995efdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:27.174870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315deeb-a83e-4261-b529-868ab995efdc None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:27.175145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315deeb-a83e-4261-b529-868ab995efdc None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:27.175409 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315deeb-a83e-4261-b529-868ab995efdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:27.185350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315deeb-a83e-4261-b529-868ab995efdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:27.186775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315deeb-a83e-4261-b529-868ab995efdc None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-8315deeb-a83e-4261-b529-868ab995efdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:09:27.201116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:27.206974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:09:27.208129 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:09:27.208129 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:27.208129 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:27.208129 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 624/1251] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:27.208409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b88f0-95e7-480f-a12c-4dbdf4995426 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:27.310531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b88f0-95e7-480f-a12c-4dbdf4995426 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:27.344059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec706dc-2621-4ebd-9f03-e18d5cedf8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:27.344692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec706dc-2621-4ebd-9f03-e18d5cedf8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:27.344974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec706dc-2621-4ebd-9f03-e18d5cedf8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:27.345248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec706dc-2621-4ebd-9f03-e18d5cedf8b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:27.345884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec706dc-2621-4ebd-9f03-e18d5cedf8b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:27.346188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec706dc-2621-4ebd-9f03-e18d5cedf8b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:27.347548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec706dc-2621-4ebd-9f03-e18d5cedf8b6 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:27.359929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 628/1252] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:09:27.365653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315deeb-a83e-4261-b529-868ab995efdc None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:27.477864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315deeb-a83e-4261-b529-868ab995efdc None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:27.504378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:27.512186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eeb3ab-bd59-456a-8e85-f296e6d77bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:27.512850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeb3ab-bd59-456a-8e85-f296e6d77bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:27.513124 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeb3ab-bd59-456a-8e85-f296e6d77bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:27.513444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeb3ab-bd59-456a-8e85-f296e6d77bd8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:27.521149 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eeb3ab-bd59-456a-8e85-f296e6d77bd8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:27.522596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eeb3ab-bd59-456a-8e85-f296e6d77bd8 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-55eeb3ab-bd59-456a-8e85-f296e6d77bd8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:09:27.534758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:27.535961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:27.536979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:27.536979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:27.536979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:09:27.536979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:09:27.536979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:09:27.536979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:09:27.536979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:09:27.536979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:09:27.536979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:09:27.536979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:27.536979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 625/1253] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:27.543130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec706dc-2621-4ebd-9f03-e18d5cedf8b6 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:27.597851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:27.641395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec706dc-2621-4ebd-9f03-e18d5cedf8b6 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:27.784967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8517c-b9bd-49fa-b13a-f89cd0075148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:27.784967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8517c-b9bd-49fa-b13a-f89cd0075148 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:27.784967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8517c-b9bd-49fa-b13a-f89cd0075148 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:27.784967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8517c-b9bd-49fa-b13a-f89cd0075148 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:27.786023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8517c-b9bd-49fa-b13a-f89cd0075148 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:27.786023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8517c-b9bd-49fa-b13a-f89cd0075148 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:27.786023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8517c-b9bd-49fa-b13a-f89cd0075148 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:27.786023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 629/1254] 10.222.0.85 () {56 vars in 1143 bytes} [Mon Aug 29 20:09:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:27.786296 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eeb3ab-bd59-456a-8e85-f296e6d77bd8 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:27.809380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eeb3ab-bd59-456a-8e85-f296e6d77bd8 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:27.842356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e4136-3ff9-4ba1-9e5f-631cefae7cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:27.842957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e4136-3ff9-4ba1-9e5f-631cefae7cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:27.843239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e4136-3ff9-4ba1-9e5f-631cefae7cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:27.843514 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e4136-3ff9-4ba1-9e5f-631cefae7cd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:27.897810 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:27.910161 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e4136-3ff9-4ba1-9e5f-631cefae7cd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:27.911613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e4136-3ff9-4ba1-9e5f-631cefae7cd5 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:28.007376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 626/1255] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:09:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:09:28.013369 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8517c-b9bd-49fa-b13a-f89cd0075148 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:28.113581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8517c-b9bd-49fa-b13a-f89cd0075148 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:28.127272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:28.148950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab0fa7-b2c0-462e-b13a-cc31033f7dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:28.152332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab0fa7-b2c0-462e-b13a-cc31033f7dcb None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:28.152332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab0fa7-b2c0-462e-b13a-cc31033f7dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:28.152332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab0fa7-b2c0-462e-b13a-cc31033f7dcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:28.159976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab0fa7-b2c0-462e-b13a-cc31033f7dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:28.161506 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab0fa7-b2c0-462e-b13a-cc31033f7dcb None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-d1ab0fa7-b2c0-462e-b13a-cc31033f7dcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:09:28.175722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:28.182624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:09:28.183837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:09:28.183837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:28.183837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:28.183837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 630/1256] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:28.184140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e4136-3ff9-4ba1-9e5f-631cefae7cd5 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:28.269130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:28.281279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e4136-3ff9-4ba1-9e5f-631cefae7cd5 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:28.315505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480de855-ac2e-47aa-bfc4-8f28e06a447a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:28.316106 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480de855-ac2e-47aa-bfc4-8f28e06a447a None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:28.316489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480de855-ac2e-47aa-bfc4-8f28e06a447a None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:28.316769 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480de855-ac2e-47aa-bfc4-8f28e06a447a None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:28.317398 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480de855-ac2e-47aa-bfc4-8f28e06a447a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:28.317700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480de855-ac2e-47aa-bfc4-8f28e06a447a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:28.319117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480de855-ac2e-47aa-bfc4-8f28e06a447a None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:28.330310 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 627/1257] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:28.336247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab0fa7-b2c0-462e-b13a-cc31033f7dcb None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:28.442107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab0fa7-b2c0-462e-b13a-cc31033f7dcb None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:28.476563 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1eeda-ab8c-459d-9eee-4e18e72d8b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:28.477188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1eeda-ab8c-459d-9eee-4e18e72d8b5b None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:28.477469 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1eeda-ab8c-459d-9eee-4e18e72d8b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:28.477743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1eeda-ab8c-459d-9eee-4e18e72d8b5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:28.484271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1eeda-ab8c-459d-9eee-4e18e72d8b5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:28.485698 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1eeda-ab8c-459d-9eee-4e18e72d8b5b None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-a4f1eeda-ab8c-459d-9eee-4e18e72d8b5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:09:28.496666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:28.497677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:28.498814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:09:28.498814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:09:28.498814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:09:28.498814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:09:28.498814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:09:28.498814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:09:28.498814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:09:28.498814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:28.498814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 631/1258] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:09:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:28.504081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480de855-ac2e-47aa-bfc4-8f28e06a447a None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:28.606399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480de855-ac2e-47aa-bfc4-8f28e06a447a None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:28.640053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df5718-3b13-4c27-b6fb-05f42a381b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:28.640679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df5718-3b13-4c27-b6fb-05f42a381b03 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:28.640971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df5718-3b13-4c27-b6fb-05f42a381b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:28.641223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df5718-3b13-4c27-b6fb-05f42a381b03 None admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:28.641903 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df5718-3b13-4c27-b6fb-05f42a381b03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:28.642204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df5718-3b13-4c27-b6fb-05f42a381b03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:28.643617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df5718-3b13-4c27-b6fb-05f42a381b03 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:28.656028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 628/1259] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:09:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:09:28.663059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1eeda-ab8c-459d-9eee-4e18e72d8b5b None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:28.688596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:28.760721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1eeda-ab8c-459d-9eee-4e18e72d8b5b None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:28.794382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40225d72-ce24-4db9-b39c-d9880ee27e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:28.794982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40225d72-ce24-4db9-b39c-d9880ee27e5b None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:28.795263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40225d72-ce24-4db9-b39c-d9880ee27e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:28.795534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40225d72-ce24-4db9-b39c-d9880ee27e5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:28.796279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40225d72-ce24-4db9-b39c-d9880ee27e5b None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=5288af2dbc15438c8b209b0ba2454cf1) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:28.796592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40225d72-ce24-4db9-b39c-d9880ee27e5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:28.798020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40225d72-ce24-4db9-b39c-d9880ee27e5b None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:28.803307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 632/1260] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:09:28 2022] GET /identity/v3/role_assignments?user.id=5288af2dbc15438c8b209b0ba2454cf1&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:30.684333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc4926-e02a-4574-886d-a809e26d2372 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:30.684920 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc4926-e02a-4574-886d-a809e26d2372 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:30.684920 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc4926-e02a-4574-886d-a809e26d2372 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:30.686203 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 629/1261] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:09:30.691336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa91948-14c0-4365-a1a8-5513a0bc42bc None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:30.691820 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa91948-14c0-4365-a1a8-5513a0bc42bc None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:30.691930 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa91948-14c0-4365-a1a8-5513a0bc42bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:30.711174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-8aa91948-14c0-4365-a1a8-5513a0bc42bc None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:09:30.712404 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8aa91948-14c0-4365-a1a8-5513a0bc42bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:09:30.777802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 633/1262] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:09:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:09:31.047121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc4926-e02a-4574-886d-a809e26d2372 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:31.059007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:31.153796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc4926-e02a-4574-886d-a809e26d2372 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:31.188911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b47c6-04d2-4cbf-8191-ac4d86a8d883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:31.189531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b47c6-04d2-4cbf-8191-ac4d86a8d883 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:31.189810 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b47c6-04d2-4cbf-8191-ac4d86a8d883 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:31.190086 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b47c6-04d2-4cbf-8191-ac4d86a8d883 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:31.200781 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b47c6-04d2-4cbf-8191-ac4d86a8d883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-d50b47c6-04d2-4cbf-8191-ac4d86a8d883 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:09:31.205624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:31.206738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:31.207877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:31.207877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:31.207877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:31.207877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:31.207877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:09:31.207877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:09:31.207877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:09:31.207877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:09:31.207877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:09:31.207877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:09:31.207877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:09:31.207877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:31.207877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 630/1263] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:09:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:31.213157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa91948-14c0-4365-a1a8-5513a0bc42bc None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:31.231875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:31.315245 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa91948-14c0-4365-a1a8-5513a0bc42bc None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:31.347733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a62cb2-9251-43a7-a937-9a4bfbbb54c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:31.348378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a62cb2-9251-43a7-a937-9a4bfbbb54c2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:31.348653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a62cb2-9251-43a7-a937-9a4bfbbb54c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:31.348928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a62cb2-9251-43a7-a937-9a4bfbbb54c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:31.349593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a62cb2-9251-43a7-a937-9a4bfbbb54c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:31.350407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a62cb2-9251-43a7-a937-9a4bfbbb54c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:31.351784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a62cb2-9251-43a7-a937-9a4bfbbb54c2 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:31.359730 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 634/1264] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:09:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:31.366226 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b47c6-04d2-4cbf-8191-ac4d86a8d883 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:31.432754 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:31.468375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b47c6-04d2-4cbf-8191-ac4d86a8d883 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:31.503401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48963324-2e3c-4c07-84d7-62936e8eaff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:31.504025 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48963324-2e3c-4c07-84d7-62936e8eaff2 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:31.504345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48963324-2e3c-4c07-84d7-62936e8eaff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:31.504679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48963324-2e3c-4c07-84d7-62936e8eaff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:31.514481 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48963324-2e3c-4c07-84d7-62936e8eaff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:31.515902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48963324-2e3c-4c07-84d7-62936e8eaff2 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-48963324-2e3c-4c07-84d7-62936e8eaff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:09:31.531299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:31.532584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:09:31.533541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:09:31.533541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:31.533541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:31.533541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 631/1265] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:31.539499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a62cb2-9251-43a7-a937-9a4bfbbb54c2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:31.622923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:31.641743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a62cb2-9251-43a7-a937-9a4bfbbb54c2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:31.675100 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09393277-a5fd-4640-a9e1-94b383460ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:31.675700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09393277-a5fd-4640-a9e1-94b383460ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:31.675976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09393277-a5fd-4640-a9e1-94b383460ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:31.676350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09393277-a5fd-4640-a9e1-94b383460ef0 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:31.676988 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09393277-a5fd-4640-a9e1-94b383460ef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:31.677286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09393277-a5fd-4640-a9e1-94b383460ef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:31.678655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09393277-a5fd-4640-a9e1-94b383460ef0 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:31.690566 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 635/1266] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:31.696511 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48963324-2e3c-4c07-84d7-62936e8eaff2 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:31.797423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48963324-2e3c-4c07-84d7-62936e8eaff2 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:31.816925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:31.831145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c619d-cc2a-429c-8245-876a7f736506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:31.831766 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c619d-cc2a-429c-8245-876a7f736506 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:31.832039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c619d-cc2a-429c-8245-876a7f736506 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:31.832336 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c619d-cc2a-429c-8245-876a7f736506 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:31.840598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c619d-cc2a-429c-8245-876a7f736506 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:31.842076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c619d-cc2a-429c-8245-876a7f736506 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-c17c619d-cc2a-429c-8245-876a7f736506 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:09:31.853568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:31.854551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:31.855806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:31.855806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:31.855806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:09:31.855806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:09:31.855806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:09:31.855806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:09:31.855806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:09:31.855806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:09:31.855806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:09:31.855806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:31.855806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 632/1267] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:31.860709 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09393277-a5fd-4640-a9e1-94b383460ef0 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:31.957357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09393277-a5fd-4640-a9e1-94b383460ef0 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:31.990135 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bc886-04f9-4941-bb4f-72403d36b029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:31.991310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bc886-04f9-4941-bb4f-72403d36b029 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:31.991586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bc886-04f9-4941-bb4f-72403d36b029 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:31.991855 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bc886-04f9-4941-bb4f-72403d36b029 None admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:31.992610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bc886-04f9-4941-bb4f-72403d36b029 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:31.992914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bc886-04f9-4941-bb4f-72403d36b029 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:31.994329 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bc886-04f9-4941-bb4f-72403d36b029 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:32.002685 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:32.011445 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 636/1268] 10.222.0.85 () {56 vars in 1143 bytes} [Mon Aug 29 20:09:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:32.017287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c619d-cc2a-429c-8245-876a7f736506 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:32.117703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c619d-cc2a-429c-8245-876a7f736506 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:32.151237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ea355-aa1c-46cf-83bb-7deb0753b263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:32.151855 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ea355-aa1c-46cf-83bb-7deb0753b263 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:32.152132 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ea355-aa1c-46cf-83bb-7deb0753b263 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:32.152428 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ea355-aa1c-46cf-83bb-7deb0753b263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:32.162707 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ea355-aa1c-46cf-83bb-7deb0753b263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:32.164206 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ea355-aa1c-46cf-83bb-7deb0753b263 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-599ea355-aa1c-46cf-83bb-7deb0753b263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:09:32.178267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:32.179394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:09:32.180497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:09:32.180497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:32.180497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:32.180497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 633/1269] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:32.185379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bc886-04f9-4941-bb4f-72403d36b029 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:32.284370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bc886-04f9-4941-bb4f-72403d36b029 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:32.316684 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ae852-d7d2-41e5-a9a2-93e270a4407e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:32.317278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ae852-d7d2-41e5-a9a2-93e270a4407e None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:32.317551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ae852-d7d2-41e5-a9a2-93e270a4407e None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:32.317816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ae852-d7d2-41e5-a9a2-93e270a4407e None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:32.318483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ae852-d7d2-41e5-a9a2-93e270a4407e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:32.319305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ae852-d7d2-41e5-a9a2-93e270a4407e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:32.320681 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ae852-d7d2-41e5-a9a2-93e270a4407e None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:32.331410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 637/1270] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:32.337169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ea355-aa1c-46cf-83bb-7deb0753b263 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:32.368865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:32.436393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ea355-aa1c-46cf-83bb-7deb0753b263 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:32.469256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b52d9-17a5-47a9-8c6a-9ed73458af89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:32.469872 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b52d9-17a5-47a9-8c6a-9ed73458af89 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:32.470147 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b52d9-17a5-47a9-8c6a-9ed73458af89 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:32.470415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b52d9-17a5-47a9-8c6a-9ed73458af89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:32.477367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b52d9-17a5-47a9-8c6a-9ed73458af89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:32.478801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b52d9-17a5-47a9-8c6a-9ed73458af89 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-a94b52d9-17a5-47a9-8c6a-9ed73458af89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:09:32.491907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:32.493002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:32.494069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:09:32.494069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:09:32.494069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:09:32.494069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:09:32.494069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:09:32.494069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:09:32.494069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:09:32.494069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:32.494069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 634/1271] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:09:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:32.499099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ae852-d7d2-41e5-a9a2-93e270a4407e None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:32.559463 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:32.600550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ae852-d7d2-41e5-a9a2-93e270a4407e None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:32.635218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a6a07-6b01-4ec2-ab70-4461d1ccb3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:32.638249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a6a07-6b01-4ec2-ab70-4461d1ccb3bd None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:32.638249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a6a07-6b01-4ec2-ab70-4461d1ccb3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:32.638249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a6a07-6b01-4ec2-ab70-4461d1ccb3bd None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:32.638249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a6a07-6b01-4ec2-ab70-4461d1ccb3bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:32.638249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a6a07-6b01-4ec2-ab70-4461d1ccb3bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:32.638831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a6a07-6b01-4ec2-ab70-4461d1ccb3bd None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:32.652560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 638/1272] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:09:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:32.658652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b52d9-17a5-47a9-8c6a-9ed73458af89 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:32.748187 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:32.758978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b52d9-17a5-47a9-8c6a-9ed73458af89 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:32.791913 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822debf4-0065-4075-94be-8a98b2cb9643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:32.792641 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822debf4-0065-4075-94be-8a98b2cb9643 None admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:32.792990 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822debf4-0065-4075-94be-8a98b2cb9643 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:32.793295 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822debf4-0065-4075-94be-8a98b2cb9643 None admin] PATH_INFO: `/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/5288af2dbc15438c8b209b0ba2454cf1/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:32.795259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822debf4-0065-4075-94be-8a98b2cb9643 None admin] RBAC: Authorizing `identity:create_grant(project_id=11372fe8994543b4a19773c0e0bee53e, user_id=5288af2dbc15438c8b209b0ba2454cf1, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:32.796799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822debf4-0065-4075-94be-8a98b2cb9643 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:32.912038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 635/1273] 10.222.0.85 () {58 vars in 1439 bytes} [Mon Aug 29 20:09:32 2022] PUT /identity/v3/projects/11372fe8994543b4a19773c0e0bee53e/users/5288af2dbc15438c8b209b0ba2454cf1/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:09:34.827166 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c72d7-d3c2-4d6a-a7f9-862ffe077c3f None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:34.827694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c72d7-d3c2-4d6a-a7f9-862ffe077c3f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:34.827856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c72d7-d3c2-4d6a-a7f9-862ffe077c3f None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:34.829235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 639/1274] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:09:34.834413 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a63124-5fc3-4388-909c-88fe9a501a2b None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:34.834822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a63124-5fc3-4388-909c-88fe9a501a2b None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:34.836559 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a63124-5fc3-4388-909c-88fe9a501a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:34.853951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b7a63124-5fc3-4388-909c-88fe9a501a2b None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:09:34.855123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b7a63124-5fc3-4388-909c-88fe9a501a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:09:34.923097 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 636/1275] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:09:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:09:35.189396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c72d7-d3c2-4d6a-a7f9-862ffe077c3f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:35.289024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c72d7-d3c2-4d6a-a7f9-862ffe077c3f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:35.305702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:35.323930 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c1e25-0100-4de7-8ccb-707ef8c687f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:35.324636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c1e25-0100-4de7-8ccb-707ef8c687f7 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:35.324944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c1e25-0100-4de7-8ccb-707ef8c687f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:35.325213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c1e25-0100-4de7-8ccb-707ef8c687f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:35.336088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c1e25-0100-4de7-8ccb-707ef8c687f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-360c1e25-0100-4de7-8ccb-707ef8c687f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:09:35.341027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:35.342163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:35.343254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:35.343254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:35.343254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:35.343254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:35.343254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:09:35.343254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:09:35.343254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:09:35.343254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:09:35.343254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:09:35.343254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:09:35.343254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:09:35.343254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:35.343254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 640/1276] 10.222.0.85 () {56 vars in 1096 bytes} [Mon Aug 29 20:09:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:35.348314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a63124-5fc3-4388-909c-88fe9a501a2b None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:35.429472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:35.448549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a63124-5fc3-4388-909c-88fe9a501a2b None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:35.480735 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3c32d-8b34-4743-8964-4782a4a5bc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:35.481333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3c32d-8b34-4743-8964-4782a4a5bc76 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:35.481609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3c32d-8b34-4743-8964-4782a4a5bc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:35.481878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3c32d-8b34-4743-8964-4782a4a5bc76 None admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:35.482529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3c32d-8b34-4743-8964-4782a4a5bc76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:35.482832 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3c32d-8b34-4743-8964-4782a4a5bc76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:35.484202 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3c32d-8b34-4743-8964-4782a4a5bc76 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:35.492544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 637/1277] 10.222.0.85 () {56 vars in 1099 bytes} [Mon Aug 29 20:09:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:35.498335 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c1e25-0100-4de7-8ccb-707ef8c687f7 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:35.599539 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c1e25-0100-4de7-8ccb-707ef8c687f7 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:35.634479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c98f2-3f4a-42b0-93f0-8f2162e37ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:35.635113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c98f2-3f4a-42b0-93f0-8f2162e37ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:35.635392 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c98f2-3f4a-42b0-93f0-8f2162e37ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:35.635690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c98f2-3f4a-42b0-93f0-8f2162e37ae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:35.645628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c98f2-3f4a-42b0-93f0-8f2162e37ae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:35.647729 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c98f2-3f4a-42b0-93f0-8f2162e37ae9 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-273c98f2-3f4a-42b0-93f0-8f2162e37ae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:09:35.663560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:35.664805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:09:35.665768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:09:35.665768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:35.665768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:35.665768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 641/1278] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:35.670919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3c32d-8b34-4743-8964-4782a4a5bc76 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:35.771530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3c32d-8b34-4743-8964-4782a4a5bc76 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:35.805027 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280761e5-3523-43f4-b4aa-2b8fd1ccd56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:35.805612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280761e5-3523-43f4-b4aa-2b8fd1ccd56b None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:35.805890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280761e5-3523-43f4-b4aa-2b8fd1ccd56b None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:35.806158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280761e5-3523-43f4-b4aa-2b8fd1ccd56b None admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:35.806784 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280761e5-3523-43f4-b4aa-2b8fd1ccd56b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:35.807120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280761e5-3523-43f4-b4aa-2b8fd1ccd56b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:35.808539 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280761e5-3523-43f4-b4aa-2b8fd1ccd56b None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:35.820311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 638/1279] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:35.826072 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c98f2-3f4a-42b0-93f0-8f2162e37ae9 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:35.857825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:35.933980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c98f2-3f4a-42b0-93f0-8f2162e37ae9 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:35.966354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30461999-4509-44e6-8f7c-af0b9184fbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:35.966974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30461999-4509-44e6-8f7c-af0b9184fbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:35.967255 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30461999-4509-44e6-8f7c-af0b9184fbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:35.967571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30461999-4509-44e6-8f7c-af0b9184fbc3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:35.975294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30461999-4509-44e6-8f7c-af0b9184fbc3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:35.976769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30461999-4509-44e6-8f7c-af0b9184fbc3 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-30461999-4509-44e6-8f7c-af0b9184fbc3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:09:35.987677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:35.994159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:35.995396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:35.995396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:35.995396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:09:35.995396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:09:35.995396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:09:35.995396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:09:35.995396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:09:35.995396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:09:35.995396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:09:35.995396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:35.995396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 642/1280] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:35.996729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280761e5-3523-43f4-b4aa-2b8fd1ccd56b None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:36.000523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:36.095681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280761e5-3523-43f4-b4aa-2b8fd1ccd56b None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:36.130474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9eb118-af52-4fd4-bf31-9d058671ca91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:36.131712 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eb118-af52-4fd4-bf31-9d058671ca91 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:36.131987 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eb118-af52-4fd4-bf31-9d058671ca91 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:36.132329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eb118-af52-4fd4-bf31-9d058671ca91 None admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:36.133053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9eb118-af52-4fd4-bf31-9d058671ca91 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:36.133502 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9eb118-af52-4fd4-bf31-9d058671ca91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:36.135184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9eb118-af52-4fd4-bf31-9d058671ca91 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:36.152795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 639/1281] 10.222.0.85 () {56 vars in 1143 bytes} [Mon Aug 29 20:09:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:09:36.158702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30461999-4509-44e6-8f7c-af0b9184fbc3 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:36.250801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:36.260747 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30461999-4509-44e6-8f7c-af0b9184fbc3 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:36.295769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438808ef-4fe5-4749-879d-17a58eefc1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:36.296403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438808ef-4fe5-4749-879d-17a58eefc1ac None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:36.296683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438808ef-4fe5-4749-879d-17a58eefc1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:36.296956 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438808ef-4fe5-4749-879d-17a58eefc1ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:36.364359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438808ef-4fe5-4749-879d-17a58eefc1ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:36.366304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438808ef-4fe5-4749-879d-17a58eefc1ac None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:36.453193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:36.463160 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 643/1282] 10.222.0.85 () {58 vars in 1282 bytes} [Mon Aug 29 20:09:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:09:36.469237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9eb118-af52-4fd4-bf31-9d058671ca91 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:36.530063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:36.567306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9eb118-af52-4fd4-bf31-9d058671ca91 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:36.601643 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61939f29-fe6f-4490-875c-cb70bb69c2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:36.602279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61939f29-fe6f-4490-875c-cb70bb69c2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:36.602560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61939f29-fe6f-4490-875c-cb70bb69c2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:36.602836 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61939f29-fe6f-4490-875c-cb70bb69c2c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:36.613625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61939f29-fe6f-4490-875c-cb70bb69c2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:36.615181 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61939f29-fe6f-4490-875c-cb70bb69c2c1 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-61939f29-fe6f-4490-875c-cb70bb69c2c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:09:36.630443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:36.631680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:09:36.632790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:09:36.632790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:09:36.632790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:36.632790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 640/1283] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:36.637239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438808ef-4fe5-4749-879d-17a58eefc1ac None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:36.738510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438808ef-4fe5-4749-879d-17a58eefc1ac None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:36.772270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f8b7c-7787-45d6-9e92-f9c5bd095f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:36.774994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f8b7c-7787-45d6-9e92-f9c5bd095f42 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:36.774994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f8b7c-7787-45d6-9e92-f9c5bd095f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:36.774994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f8b7c-7787-45d6-9e92-f9c5bd095f42 None admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:36.774994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f8b7c-7787-45d6-9e92-f9c5bd095f42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:36.774994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f8b7c-7787-45d6-9e92-f9c5bd095f42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:36.775951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f8b7c-7787-45d6-9e92-f9c5bd095f42 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:36.787185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 644/1284] 10.222.0.85 () {56 vars in 1109 bytes} [Mon Aug 29 20:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:36.792889 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61939f29-fe6f-4490-875c-cb70bb69c2c1 None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:36.891216 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61939f29-fe6f-4490-875c-cb70bb69c2c1 None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:36.904580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:36.924113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d1d8c-3a58-43d2-bdbf-50df017ddcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:36.924778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d1d8c-3a58-43d2-bdbf-50df017ddcac None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:36.925091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d1d8c-3a58-43d2-bdbf-50df017ddcac None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:36.925355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d1d8c-3a58-43d2-bdbf-50df017ddcac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:36.932610 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d1d8c-3a58-43d2-bdbf-50df017ddcac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:36.934149 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d1d8c-3a58-43d2-bdbf-50df017ddcac None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application [None req-0b0d1d8c-3a58-43d2-bdbf-50df017ddcac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:09:36.946234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:36.947363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:36.948749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:09:36.948749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:09:36.948749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:09:36.948749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:09:36.948749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:09:36.948749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:09:36.948749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:09:36.948749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: ERROR keystone.server.flask.application Aug 29 20:09:36.948749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 641/1285] 10.222.0.85 () {56 vars in 1111 bytes} [Mon Aug 29 20:09:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:36.953256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f8b7c-7787-45d6-9e92-f9c5bd095f42 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:37.002869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:37.051374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f8b7c-7787-45d6-9e92-f9c5bd095f42 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:37.084744 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7901773-0c8c-40d1-9615-6f678db23786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:37.085342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7901773-0c8c-40d1-9615-6f678db23786 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:37.085632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7901773-0c8c-40d1-9615-6f678db23786 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:37.085880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7901773-0c8c-40d1-9615-6f678db23786 None admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:37.086560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7901773-0c8c-40d1-9615-6f678db23786 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:37.086890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7901773-0c8c-40d1-9615-6f678db23786 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:37.088448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7901773-0c8c-40d1-9615-6f678db23786 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:37.101000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 645/1286] 10.222.0.85 () {56 vars in 1148 bytes} [Mon Aug 29 20:09:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:37.107136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d1d8c-3a58-43d2-bdbf-50df017ddcac None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:37.210680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d1d8c-3a58-43d2-bdbf-50df017ddcac None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:37.242909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f400c26-0992-49ea-b2ef-c0c3d48edc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:37.243505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f400c26-0992-49ea-b2ef-c0c3d48edc6d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:37.243787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f400c26-0992-49ea-b2ef-c0c3d48edc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:37.244062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f400c26-0992-49ea-b2ef-c0c3d48edc6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:37.244819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f400c26-0992-49ea-b2ef-c0c3d48edc6d None admin] RBAC: Adding query filter params (role.id=58f6b4d2e3664607851e63c89b5ca3d3, scope.project.id=11372fe8994543b4a19773c0e0bee53e, user.id=5288af2dbc15438c8b209b0ba2454cf1) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:37.245132 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f400c26-0992-49ea-b2ef-c0c3d48edc6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:37.246564 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f400c26-0992-49ea-b2ef-c0c3d48edc6d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:37.252268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 642/1287] 10.222.0.85 () {56 vars in 1374 bytes} [Mon Aug 29 20:09:37 2022] GET /identity/v3/role_assignments?user.id=5288af2dbc15438c8b209b0ba2454cf1&scope.project.id=11372fe8994543b4a19773c0e0bee53e&role.id=58f6b4d2e3664607851e63c89b5ca3d3 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:39.205374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e1b36-d97a-45f5-a8ec-18a0a86288e0 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:39.205910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e1b36-d97a-45f5-a8ec-18a0a86288e0 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:39.205910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e1b36-d97a-45f5-a8ec-18a0a86288e0 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:39.207256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 646/1288] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:09:39.213138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980a233-f7d6-47f9-826f-5516e38b468e None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:39.218646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980a233-f7d6-47f9-826f-5516e38b468e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:39.218646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980a233-f7d6-47f9-826f-5516e38b468e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:39.232535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b980a233-f7d6-47f9-826f-5516e38b468e None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:09:39.233727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b980a233-f7d6-47f9-826f-5516e38b468e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:09:39.263016 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:39.300637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 643/1289] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:09:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:09:39.570989 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e1b36-d97a-45f5-a8ec-18a0a86288e0 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:39.673934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e1b36-d97a-45f5-a8ec-18a0a86288e0 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:39.685365 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:39.707665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d0bdd-c5d1-4cf9-b32d-96bfa509a5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:39.708321 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d0bdd-c5d1-4cf9-b32d-96bfa509a5fe None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:39.708605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d0bdd-c5d1-4cf9-b32d-96bfa509a5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:39.709397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d0bdd-c5d1-4cf9-b32d-96bfa509a5fe None admin] PATH_INFO: `/v3/services/placement` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:39.709977 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d0bdd-c5d1-4cf9-b32d-96bfa509a5fe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:39.711376 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d0bdd-c5d1-4cf9-b32d-96bfa509a5fe None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-c91d0bdd-c5d1-4cf9-b32d-96bfa509a5fe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:39.719957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:39.723752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:39.724913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 647/1290] 10.222.0.85 () {56 vars in 1117 bytes} [Mon Aug 29 20:09:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:39.728465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980a233-f7d6-47f9-826f-5516e38b468e None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:39.834630 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980a233-f7d6-47f9-826f-5516e38b468e None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:39.868953 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c0e54-5f71-4a3f-a89f-fc6459bd492f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:39.869553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c0e54-5f71-4a3f-a89f-fc6459bd492f None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:39.869831 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c0e54-5f71-4a3f-a89f-fc6459bd492f None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:39.870103 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c0e54-5f71-4a3f-a89f-fc6459bd492f None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:39.870735 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c0e54-5f71-4a3f-a89f-fc6459bd492f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:39.871036 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c0e54-5f71-4a3f-a89f-fc6459bd492f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:39.872457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c0e54-5f71-4a3f-a89f-fc6459bd492f None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:39.877143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 644/1291] 10.222.0.85 () {56 vars in 1116 bytes} [Mon Aug 29 20:09:39 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:39.883199 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d0bdd-c5d1-4cf9-b32d-96bfa509a5fe None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:39.982859 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d0bdd-c5d1-4cf9-b32d-96bfa509a5fe None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:40.016015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dbc79-6a64-4a5b-9918-70ae7bf81650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:40.016645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dbc79-6a64-4a5b-9918-70ae7bf81650 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:40.016924 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dbc79-6a64-4a5b-9918-70ae7bf81650 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:40.017197 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dbc79-6a64-4a5b-9918-70ae7bf81650 None admin] PATH_INFO: `/v3/services` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:40.017818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dbc79-6a64-4a5b-9918-70ae7bf81650 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:40.018119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dbc79-6a64-4a5b-9918-70ae7bf81650 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:40.019474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dbc79-6a64-4a5b-9918-70ae7bf81650 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:40.024056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 648/1292] 10.222.0.85 () {56 vars in 1116 bytes} [Mon Aug 29 20:09:39 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:42.989770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21062a5c-0e19-4af5-91c6-d0a58c04dafd None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:42.990934 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21062a5c-0e19-4af5-91c6-d0a58c04dafd None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:42.990934 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21062a5c-0e19-4af5-91c6-d0a58c04dafd None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:42.992276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 645/1293] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:09:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:09:42.997400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b3842-63d3-4c26-bede-0b552449da6f None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:42.997846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b3842-63d3-4c26-bede-0b552449da6f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:42.997959 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b3842-63d3-4c26-bede-0b552449da6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:43.016734 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-627b3842-63d3-4c26-bede-0b552449da6f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:09:43.018781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-627b3842-63d3-4c26-bede-0b552449da6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:09:43.046835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:43.083430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 649/1294] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:09:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:09:43.352582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21062a5c-0e19-4af5-91c6-d0a58c04dafd None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:43.385146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:43.452668 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21062a5c-0e19-4af5-91c6-d0a58c04dafd None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:43.486010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf5cfa-e7d8-4cf3-b60f-9a5e5eb22a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:43.486609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf5cfa-e7d8-4cf3-b60f-9a5e5eb22a18 None admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:43.486887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf5cfa-e7d8-4cf3-b60f-9a5e5eb22a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:43.487157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf5cfa-e7d8-4cf3-b60f-9a5e5eb22a18 None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:43.487825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf5cfa-e7d8-4cf3-b60f-9a5e5eb22a18 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:43.489225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf5cfa-e7d8-4cf3-b60f-9a5e5eb22a18 None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:43.620559 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 646/1295] 10.222.0.85 () {60 vars in 1141 bytes} [Mon Aug 29 20:09:43 2022] POST /identity/v3/services => generated 246 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:09:45.557659 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67b7f8-d79c-4f38-9083-2228645310d0 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:45.558269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67b7f8-d79c-4f38-9083-2228645310d0 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:45.558269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67b7f8-d79c-4f38-9083-2228645310d0 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:45.559551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 650/1296] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:09:45.565404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209a693-ed29-47dd-8f65-b82dc537dc78 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:45.566851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209a693-ed29-47dd-8f65-b82dc537dc78 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:45.566851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209a693-ed29-47dd-8f65-b82dc537dc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:45.585893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3209a693-ed29-47dd-8f65-b82dc537dc78 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:09:45.587102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3209a693-ed29-47dd-8f65-b82dc537dc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:09:45.670856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 647/1297] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:09:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:09:45.941864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67b7f8-d79c-4f38-9083-2228645310d0 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:46.043679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67b7f8-d79c-4f38-9083-2228645310d0 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:46.076864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06283e6-1dbe-403f-b581-0663db2db126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:46.077482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06283e6-1dbe-403f-b581-0663db2db126 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:46.077762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06283e6-1dbe-403f-b581-0663db2db126 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:46.078034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06283e6-1dbe-403f-b581-0663db2db126 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:46.078604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06283e6-1dbe-403f-b581-0663db2db126 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:46.079950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06283e6-1dbe-403f-b581-0663db2db126 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-d06283e6-1dbe-403f-b581-0663db2db126 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:46.088586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:46.089796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:46.090799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 651/1298] 10.222.0.85 () {56 vars in 1117 bytes} [Mon Aug 29 20:09:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:46.095851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209a693-ed29-47dd-8f65-b82dc537dc78 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:46.114822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:46.194428 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209a693-ed29-47dd-8f65-b82dc537dc78 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:46.228849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf35b5-1f20-4d53-847e-2bb461de20de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:46.229453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf35b5-1f20-4d53-847e-2bb461de20de None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:46.230493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf35b5-1f20-4d53-847e-2bb461de20de None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:46.230493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf35b5-1f20-4d53-847e-2bb461de20de None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:46.230712 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf35b5-1f20-4d53-847e-2bb461de20de None admin] RBAC: Adding query filter params (name=placement) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:46.230991 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf35b5-1f20-4d53-847e-2bb461de20de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:46.232391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf35b5-1f20-4d53-847e-2bb461de20de None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:46.237354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 648/1299] 10.222.0.85 () {56 vars in 1116 bytes} [Mon Aug 29 20:09:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:46.243427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06283e6-1dbe-403f-b581-0663db2db126 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:46.275279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:46.341474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06283e6-1dbe-403f-b581-0663db2db126 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:46.374657 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f694899-3810-4a6e-8fbb-041ce4c94a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:46.375263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f694899-3810-4a6e-8fbb-041ce4c94a44 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:46.375543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f694899-3810-4a6e-8fbb-041ce4c94a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:46.375815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f694899-3810-4a6e-8fbb-041ce4c94a44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:46.376550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f694899-3810-4a6e-8fbb-041ce4c94a44 None admin] RBAC: Adding query filter params (interface=public, service_id=7e6ba247f26f4cedb511876fae121fd3, region_id=RegionOne) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:46.376852 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f694899-3810-4a6e-8fbb-041ce4c94a44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:46.378365 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f694899-3810-4a6e-8fbb-041ce4c94a44 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:46.383481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 652/1300] 10.222.0.85 () {56 vars in 1251 bytes} [Mon Aug 29 20:09:46 2022] GET /identity/v3/endpoints?service_id=7e6ba247f26f4cedb511876fae121fd3&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:46.389155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf35b5-1f20-4d53-847e-2bb461de20de None admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:46.470586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:46.491426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf35b5-1f20-4d53-847e-2bb461de20de None admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:46.526182 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a95c8-2385-400a-8010-d7e1f49079af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:46.526780 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a95c8-2385-400a-8010-d7e1f49079af None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:46.527058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a95c8-2385-400a-8010-d7e1f49079af None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:46.527328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a95c8-2385-400a-8010-d7e1f49079af None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:46.527924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a95c8-2385-400a-8010-d7e1f49079af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:46.529305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a95c8-2385-400a-8010-d7e1f49079af None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:46.536747 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 649/1301] 10.222.0.85 () {56 vars in 1087 bytes} [Mon Aug 29 20:09:46 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:09:48.432575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21802012-cdd8-4952-b6a3-04fe9bb3e3ba None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:48.433115 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21802012-cdd8-4952-b6a3-04fe9bb3e3ba None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:48.433115 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21802012-cdd8-4952-b6a3-04fe9bb3e3ba None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:48.434464 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 653/1302] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:09:48.439502 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b03b2-9832-4116-93ef-695045867f58 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:48.439904 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b03b2-9832-4116-93ef-695045867f58 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:48.440046 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b03b2-9832-4116-93ef-695045867f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:48.458558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ba2b03b2-9832-4116-93ef-695045867f58 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:09:48.459714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ba2b03b2-9832-4116-93ef-695045867f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:09:48.526723 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 650/1303] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:09:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:09:48.795514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21802012-cdd8-4952-b6a3-04fe9bb3e3ba None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:48.893231 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:48.903417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21802012-cdd8-4952-b6a3-04fe9bb3e3ba None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:48.937250 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e280f5-d53f-4b52-bfcb-af33223560b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:48.937879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e280f5-d53f-4b52-bfcb-af33223560b2 None admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:48.938154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e280f5-d53f-4b52-bfcb-af33223560b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:48.938421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e280f5-d53f-4b52-bfcb-af33223560b2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:48.939073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e280f5-d53f-4b52-bfcb-af33223560b2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:48.940528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e280f5-d53f-4b52-bfcb-af33223560b2 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application [None req-f4e280f5-d53f-4b52-bfcb-af33223560b2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:09:48.950058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:09:48.951261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application with Lock( Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: ERROR keystone.server.flask.application Aug 29 20:09:48.952145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 654/1304] 10.222.0.85 () {56 vars in 1117 bytes} [Mon Aug 29 20:09:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:09:48.957566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b03b2-9832-4116-93ef-695045867f58 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:49.033229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:49.055065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b03b2-9832-4116-93ef-695045867f58 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:49.090491 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7892ff-7d70-4d27-8a91-802b571f3a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:49.091098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7892ff-7d70-4d27-8a91-802b571f3a5d None admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:49.091389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7892ff-7d70-4d27-8a91-802b571f3a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:49.091718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7892ff-7d70-4d27-8a91-802b571f3a5d None admin] PATH_INFO: `/v3/services` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:49.092378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7892ff-7d70-4d27-8a91-802b571f3a5d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:09:49.092773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7892ff-7d70-4d27-8a91-802b571f3a5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:49.094127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7892ff-7d70-4d27-8a91-802b571f3a5d None admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:49.099184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 651/1305] 10.222.0.85 () {56 vars in 1116 bytes} [Mon Aug 29 20:09:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:09:49.105202 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e280f5-d53f-4b52-bfcb-af33223560b2 None admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:09:49.204746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e280f5-d53f-4b52-bfcb-af33223560b2 None admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:09:49.237489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634cc5f-b0b0-4aa8-9883-d6a1672a2244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:09:49.238090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634cc5f-b0b0-4aa8-9883-d6a1672a2244 None admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:09:49.238373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634cc5f-b0b0-4aa8-9883-d6a1672a2244 None admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:09:49.238645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634cc5f-b0b0-4aa8-9883-d6a1672a2244 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:09:49.239331 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634cc5f-b0b0-4aa8-9883-d6a1672a2244 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:09:49.241257 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634cc5f-b0b0-4aa8-9883-d6a1672a2244 None admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:09:49.328865 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 655/1306] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:09:49 2022] POST /identity/v3/endpoints => generated 331 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:09:59.096147 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=30 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:09:59.323979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:42.641236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af30b9-7563-43e1-b572-9ba4e1a2d080 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:42.641669 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af30b9-7563-43e1-b572-9ba4e1a2d080 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:42.642172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af30b9-7563-43e1-b572-9ba4e1a2d080 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:42.649333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 652/1307] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:18:42 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:42.653987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fe2d7-6ee6-44f5-8fce-bb2088967d23 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:42.654438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fe2d7-6ee6-44f5-8fce-bb2088967d23 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:42.654536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fe2d7-6ee6-44f5-8fce-bb2088967d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:42.807648 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-e78fe2d7-6ee6-44f5-8fce-bb2088967d23 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:42.810431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e78fe2d7-6ee6-44f5-8fce-bb2088967d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:42.967703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 656/1308] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:43.261508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a725e-af7c-40b6-89a8-67f3671f958e None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.261792 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a725e-af7c-40b6-89a8-67f3671f958e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.262070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a725e-af7c-40b6-89a8-67f3671f958e None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.263639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 653/1309] 10.222.0.85 () {54 vars in 965 bytes} [Mon Aug 29 20:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:43.270687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983288e4-d9aa-4f48-9056-6ed41f49b2ca None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.270974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983288e4-d9aa-4f48-9056-6ed41f49b2ca None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.271247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983288e4-d9aa-4f48-9056-6ed41f49b2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.355850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-983288e4-d9aa-4f48-9056-6ed41f49b2ca None None] MFA Rules not processed for user `aa222b1c89dc4d2cb1686b4838c7908b`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:43.357107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-983288e4-d9aa-4f48-9056-6ed41f49b2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:43.381363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:43.443611 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 657/1310] 10.222.0.85 () {58 vars in 1062 bytes} [Mon Aug 29 20:18:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:43.453450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a725e-af7c-40b6-89a8-67f3671f958e None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:43.455361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-0b1a725e-af7c-40b6-89a8-67f3671f958e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:43.566269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a725e-af7c-40b6-89a8-67f3671f958e None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:43.650846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cda02-fe85-4acc-98f6-a3b64dfc891d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:43.651704 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cda02-fe85-4acc-98f6-a3b64dfc891d service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.651997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cda02-fe85-4acc-98f6-a3b64dfc891d service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.652305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cda02-fe85-4acc-98f6-a3b64dfc891d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.654064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d89cda02-fe85-4acc-98f6-a3b64dfc891d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:43.717402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:43.778389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cda02-fe85-4acc-98f6-a3b64dfc891d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:43.780444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cda02-fe85-4acc-98f6-a3b64dfc891d service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:43.800938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 654/1311] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:46.045374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e88b23-8196-4f14-b941-f09bb05e731d None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:46.045840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e88b23-8196-4f14-b941-f09bb05e731d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:46.045965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e88b23-8196-4f14-b941-f09bb05e731d None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:46.047330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 658/1312] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:46.052428 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660d200-0fb1-4784-a264-d6fe591ff0a6 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:46.052858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660d200-0fb1-4784-a264-d6fe591ff0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:46.053093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660d200-0fb1-4784-a264-d6fe591ff0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:46.122900 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-7660d200-0fb1-4784-a264-d6fe591ff0a6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:46.124674 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7660d200-0fb1-4784-a264-d6fe591ff0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:46.223372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 655/1313] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:46.501426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e88b23-8196-4f14-b941-f09bb05e731d None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:46.502950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c4e88b23-8196-4f14-b941-f09bb05e731d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:46.606501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e88b23-8196-4f14-b941-f09bb05e731d None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:46.677289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:46.679798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2c7bf-c561-4b2d-a5d6-e4ea0910b403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:46.680429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2c7bf-c561-4b2d-a5d6-e4ea0910b403 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:46.680710 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2c7bf-c561-4b2d-a5d6-e4ea0910b403 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:46.680974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2c7bf-c561-4b2d-a5d6-e4ea0910b403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:46.682565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-6ab2c7bf-c561-4b2d-a5d6-e4ea0910b403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:46.788479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2c7bf-c561-4b2d-a5d6-e4ea0910b403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:46.789828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2c7bf-c561-4b2d-a5d6-e4ea0910b403 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:46.807272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 659/1314] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:49.236437 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb52068-5e76-44a6-969d-0e7085e5450d None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:49.236975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb52068-5e76-44a6-969d-0e7085e5450d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:49.236975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb52068-5e76-44a6-969d-0e7085e5450d None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:49.238349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 656/1315] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:18:49.243449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ff461-f4c8-4ae8-bc8d-a66cbda62c98 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:49.243852 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ff461-f4c8-4ae8-bc8d-a66cbda62c98 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:49.244023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ff461-f4c8-4ae8-bc8d-a66cbda62c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:49.308004 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-fc4ff461-f4c8-4ae8-bc8d-a66cbda62c98 None None] MFA Rules not processed for user `f535a214d4d940d699e82a9e017b9c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:49.309220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-fc4ff461-f4c8-4ae8-bc8d-a66cbda62c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:49.359653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:49.404588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 660/1316] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:49 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:49.698846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb52068-5e76-44a6-969d-0e7085e5450d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:49.700408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-6eb52068-5e76-44a6-969d-0e7085e5450d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:49.710141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:49.802570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb52068-5e76-44a6-969d-0e7085e5450d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:49.874118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110cd160-111a-4bd2-9c50-452a96a1c78a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:49.874648 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cd160-111a-4bd2-9c50-452a96a1c78a service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:49.874927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cd160-111a-4bd2-9c50-452a96a1c78a service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:49.875197 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cd160-111a-4bd2-9c50-452a96a1c78a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:49.876835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-110cd160-111a-4bd2-9c50-452a96a1c78a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:49.943324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:49.992219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110cd160-111a-4bd2-9c50-452a96a1c78a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:49.993658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110cd160-111a-4bd2-9c50-452a96a1c78a service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:50.011365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 657/1317] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:49 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:52.816351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0484eae-9bed-484f-9765-36eca79570ac None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:52.816911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0484eae-9bed-484f-9765-36eca79570ac None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:52.816911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0484eae-9bed-484f-9765-36eca79570ac None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:52.818258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 661/1318] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:52.823457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138e2d0-ec8d-411f-baed-733f6be19b2d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:52.823901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138e2d0-ec8d-411f-baed-733f6be19b2d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:52.824012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138e2d0-ec8d-411f-baed-733f6be19b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:52.895027 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d138e2d0-ec8d-411f-baed-733f6be19b2d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:52.896120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d138e2d0-ec8d-411f-baed-733f6be19b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:52.993195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 658/1319] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:53.281848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0484eae-9bed-484f-9765-36eca79570ac None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:53.283378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e0484eae-9bed-484f-9765-36eca79570ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:53.383858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0484eae-9bed-484f-9765-36eca79570ac None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:53.456799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b7428-ac95-46a7-857d-a21388883953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:53.457391 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b7428-ac95-46a7-857d-a21388883953 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:53.457669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b7428-ac95-46a7-857d-a21388883953 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:53.457951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b7428-ac95-46a7-857d-a21388883953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:53.459499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-227b7428-ac95-46a7-857d-a21388883953 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:53.467640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:53.567859 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b7428-ac95-46a7-857d-a21388883953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:53.569262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b7428-ac95-46a7-857d-a21388883953 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:53.587101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 662/1320] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:56.557356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f03f47-610d-44c8-a220-c64b5d216a50 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:56.557907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f03f47-610d-44c8-a220-c64b5d216a50 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:56.557907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f03f47-610d-44c8-a220-c64b5d216a50 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:56.559340 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 659/1321] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:56.564958 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056845a6-91f8-4425-9b6a-0bde893e5d83 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:56.565393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056845a6-91f8-4425-9b6a-0bde893e5d83 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:56.565487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056845a6-91f8-4425-9b6a-0bde893e5d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:56.633242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-056845a6-91f8-4425-9b6a-0bde893e5d83 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:56.634483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-056845a6-91f8-4425-9b6a-0bde893e5d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:56.693200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:56.731814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 663/1322] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:57.012783 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f03f47-610d-44c8-a220-c64b5d216a50 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:57.014867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-16f03f47-610d-44c8-a220-c64b5d216a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:57.037383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:57.128684 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f03f47-610d-44c8-a220-c64b5d216a50 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:57.204787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a9026-ec73-4c30-bbb6-08eafaef34e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:57.205389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a9026-ec73-4c30-bbb6-08eafaef34e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:57.205665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a9026-ec73-4c30-bbb6-08eafaef34e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:57.205935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a9026-ec73-4c30-bbb6-08eafaef34e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:57.207672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ac8a9026-ec73-4c30-bbb6-08eafaef34e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:57.278527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:57.315836 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8a9026-ec73-4c30-bbb6-08eafaef34e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:57.317312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8a9026-ec73-4c30-bbb6-08eafaef34e0 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:57.336105 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 660/1323] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:18:59.293768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a7c62-f7aa-4f21-95e3-7e93b286963e None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:59.294330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a7c62-f7aa-4f21-95e3-7e93b286963e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:59.294330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a7c62-f7aa-4f21-95e3-7e93b286963e None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:59.295696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 664/1324] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:59.300974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8c972-bdc8-4a18-90cb-68c5c6af519d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:59.301453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8c972-bdc8-4a18-90cb-68c5c6af519d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:59.301564 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8c972-bdc8-4a18-90cb-68c5c6af519d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:59.369475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-2cc8c972-bdc8-4a18-90cb-68c5c6af519d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:59.370675 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2cc8c972-bdc8-4a18-90cb-68c5c6af519d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:59.464680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 661/1325] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:59.770023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981e9a-6d28-4078-bf94-7375059b6719 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:59.770408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981e9a-6d28-4078-bf94-7375059b6719 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:59.770749 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981e9a-6d28-4078-bf94-7375059b6719 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:59.772197 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 665/1326] 10.222.0.85 () {54 vars in 965 bytes} [Mon Aug 29 20:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:18:59.778190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd800c-b7e2-4048-97af-a26d178602e8 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:59.778481 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd800c-b7e2-4048-97af-a26d178602e8 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:59.778757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd800c-b7e2-4048-97af-a26d178602e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:59.810968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:59.858466 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-a3bd800c-b7e2-4048-97af-a26d178602e8 None None] MFA Rules not processed for user `aa222b1c89dc4d2cb1686b4838c7908b`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:59.859712 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a3bd800c-b7e2-4048-97af-a26d178602e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:59.941292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 662/1327] 10.222.0.85 () {58 vars in 1062 bytes} [Mon Aug 29 20:18:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:18:59.950257 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55981e9a-6d28-4078-bf94-7375059b6719 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:59.952482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-55981e9a-6d28-4078-bf94-7375059b6719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:00.065786 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55981e9a-6d28-4078-bf94-7375059b6719 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:00.140704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd37ef-6bd0-4b7c-bbb4-2d923b758be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:00.141317 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd37ef-6bd0-4b7c-bbb4-2d923b758be1 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:00.141595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd37ef-6bd0-4b7c-bbb4-2d923b758be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:00.141865 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd37ef-6bd0-4b7c-bbb4-2d923b758be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:00.143450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-98dd37ef-6bd0-4b7c-bbb4-2d923b758be1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:00.160636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:00.255428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd37ef-6bd0-4b7c-bbb4-2d923b758be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:00.256895 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd37ef-6bd0-4b7c-bbb4-2d923b758be1 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:00.275660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 666/1328] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:02.460095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a71d-ebec-45e6-8577-09b8715c3dac None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:02.460660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a71d-ebec-45e6-8577-09b8715c3dac None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:02.460660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a71d-ebec-45e6-8577-09b8715c3dac None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:02.461959 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 663/1329] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:02.467945 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8002d-e17a-44c9-824b-c7bc4bf07503 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:02.468415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8002d-e17a-44c9-824b-c7bc4bf07503 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:02.468523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8002d-e17a-44c9-824b-c7bc4bf07503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:02.534969 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-bbb8002d-e17a-44c9-824b-c7bc4bf07503 None None] MFA Rules not processed for user `f535a214d4d940d699e82a9e017b9c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:02.536205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-bbb8002d-e17a-44c9-824b-c7bc4bf07503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:02.599705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:02.646315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 667/1330] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:19:02.930671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2a71d-ebec-45e6-8577-09b8715c3dac None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:02.932263 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-64f2a71d-ebec-45e6-8577-09b8715c3dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:03.036877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2a71d-ebec-45e6-8577-09b8715c3dac None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:03.056901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:03.111230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92803f-1b00-4382-ae6f-2b57e5254a93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:03.111830 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92803f-1b00-4382-ae6f-2b57e5254a93 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:03.112109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92803f-1b00-4382-ae6f-2b57e5254a93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:03.112417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92803f-1b00-4382-ae6f-2b57e5254a93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:03.113986 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cd92803f-1b00-4382-ae6f-2b57e5254a93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:03.221162 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92803f-1b00-4382-ae6f-2b57e5254a93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:03.222642 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92803f-1b00-4382-ae6f-2b57e5254a93 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:03.241530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 664/1331] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:05.864318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e4f0c-6fd2-4692-802d-a54826862787 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:05.864852 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e4f0c-6fd2-4692-802d-a54826862787 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:05.864852 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e4f0c-6fd2-4692-802d-a54826862787 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:05.866142 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 668/1332] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:05.872297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29b97e-dbd5-4c54-8f28-a4844c07f27f None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:05.873310 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29b97e-dbd5-4c54-8f28-a4844c07f27f None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:05.873812 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29b97e-dbd5-4c54-8f28-a4844c07f27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:05.937654 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:05.940290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-1c29b97e-dbd5-4c54-8f28-a4844c07f27f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:05.941494 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-1c29b97e-dbd5-4c54-8f28-a4844c07f27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:06.035327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 665/1333] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:06.318163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e4f0c-6fd2-4692-802d-a54826862787 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:06.319768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-423e4f0c-6fd2-4692-802d-a54826862787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:06.422646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e4f0c-6fd2-4692-802d-a54826862787 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:06.495022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222d8d5-661c-4e27-8f01-3b8f3a213721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:06.495624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222d8d5-661c-4e27-8f01-3b8f3a213721 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:06.495901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222d8d5-661c-4e27-8f01-3b8f3a213721 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:06.496190 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222d8d5-661c-4e27-8f01-3b8f3a213721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:06.497806 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1222d8d5-661c-4e27-8f01-3b8f3a213721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:06.523616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:06.602527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222d8d5-661c-4e27-8f01-3b8f3a213721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:06.603957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222d8d5-661c-4e27-8f01-3b8f3a213721 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:06.623933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 669/1334] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:08.591988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0840e4e-1eab-473a-b87e-3780ce669f1e None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:08.592571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0840e4e-1eab-473a-b87e-3780ce669f1e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:08.592571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0840e4e-1eab-473a-b87e-3780ce669f1e None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:08.593854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 666/1335] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:08.599002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a2445-7970-4c63-943b-51be537e369c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:08.599410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a2445-7970-4c63-943b-51be537e369c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:08.599547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a2445-7970-4c63-943b-51be537e369c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:08.668443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-090a2445-7970-4c63-943b-51be537e369c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:08.670333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-090a2445-7970-4c63-943b-51be537e369c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:08.745510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:08.768048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 670/1336] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:09.051984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0840e4e-1eab-473a-b87e-3780ce669f1e None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:09.053553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a0840e4e-1eab-473a-b87e-3780ce669f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:09.154595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0840e4e-1eab-473a-b87e-3780ce669f1e None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:09.182205 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:09.226418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494ac5e-7642-4ae2-a9d3-464de49fd24e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:09.227014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494ac5e-7642-4ae2-a9d3-464de49fd24e service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:09.227292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494ac5e-7642-4ae2-a9d3-464de49fd24e service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:09.227604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494ac5e-7642-4ae2-a9d3-464de49fd24e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:09.229374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3494ac5e-7642-4ae2-a9d3-464de49fd24e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:09.336267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494ac5e-7642-4ae2-a9d3-464de49fd24e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:09.337634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494ac5e-7642-4ae2-a9d3-464de49fd24e service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:09.357061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 667/1337] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:12.063741 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67635e-7a75-414d-b7c4-454231eea5c0 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.064307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67635e-7a75-414d-b7c4-454231eea5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.064307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67635e-7a75-414d-b7c4-454231eea5c0 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.065652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 671/1338] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:12.070991 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d2291-5aec-4718-b02c-715fc93c09bf None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.071434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d2291-5aec-4718-b02c-715fc93c09bf None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.071541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d2291-5aec-4718-b02c-715fc93c09bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.135665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e54d2291-5aec-4718-b02c-715fc93c09bf None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:12.136879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e54d2291-5aec-4718-b02c-715fc93c09bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:12.145949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:12.232772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 668/1339] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:12.523881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67635e-7a75-414d-b7c4-454231eea5c0 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:12.525460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-fe67635e-7a75-414d-b7c4-454231eea5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:12.627664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67635e-7a75-414d-b7c4-454231eea5c0 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:12.699479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fa77c-3025-4b1e-ada1-d005cd3cd72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:12.700102 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fa77c-3025-4b1e-ada1-d005cd3cd72f service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.701007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fa77c-3025-4b1e-ada1-d005cd3cd72f service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.701281 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fa77c-3025-4b1e-ada1-d005cd3cd72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.702877 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-298fa77c-3025-4b1e-ada1-d005cd3cd72f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:12.738296 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:12.812177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fa77c-3025-4b1e-ada1-d005cd3cd72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:12.813564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fa77c-3025-4b1e-ada1-d005cd3cd72f service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:12.832007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 672/1340] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:17.190440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d2c86-7738-4dc7-a36e-94136f877de0 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:17.190975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d2c86-7738-4dc7-a36e-94136f877de0 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:17.190975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d2c86-7738-4dc7-a36e-94136f877de0 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:17.192675 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 669/1341] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:17.197863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1768f5-b0bc-415e-bf6b-276ed921a195 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:17.198325 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1768f5-b0bc-415e-bf6b-276ed921a195 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:17.198410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1768f5-b0bc-415e-bf6b-276ed921a195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:17.265596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-ec1768f5-b0bc-415e-bf6b-276ed921a195 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:17.266815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ec1768f5-b0bc-415e-bf6b-276ed921a195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:17.355974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:17.367785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 673/1342] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:17.654638 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d2c86-7738-4dc7-a36e-94136f877de0 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:17.656195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c41d2c86-7738-4dc7-a36e-94136f877de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:17.759697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d2c86-7738-4dc7-a36e-94136f877de0 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:17.799297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:17.836125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774807a-5af0-4692-a54b-30c6e9e60fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:17.836763 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774807a-5af0-4692-a54b-30c6e9e60fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:17.837044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774807a-5af0-4692-a54b-30c6e9e60fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:17.837311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774807a-5af0-4692-a54b-30c6e9e60fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:17.838916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-1774807a-5af0-4692-a54b-30c6e9e60fb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:17.948058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774807a-5af0-4692-a54b-30c6e9e60fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:17.949530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774807a-5af0-4692-a54b-30c6e9e60fb0 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:17.968435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 670/1343] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:20.981773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde08e6e-45c9-4c2b-b3fc-befe3b8326fa None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:20.982379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde08e6e-45c9-4c2b-b3fc-befe3b8326fa None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:20.982379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde08e6e-45c9-4c2b-b3fc-befe3b8326fa None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:20.983700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 674/1344] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:20.988818 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976eb85-0987-447e-b8d4-a632e7950dc9 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:20.989290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976eb85-0987-447e-b8d4-a632e7950dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:20.989396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976eb85-0987-447e-b8d4-a632e7950dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:21.056796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e976eb85-0987-447e-b8d4-a632e7950dc9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:21.058314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e976eb85-0987-447e-b8d4-a632e7950dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:21.076807 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:21.155342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 671/1345] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:21.439797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde08e6e-45c9-4c2b-b3fc-befe3b8326fa None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:21.441378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-fde08e6e-45c9-4c2b-b3fc-befe3b8326fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:21.547386 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde08e6e-45c9-4c2b-b3fc-befe3b8326fa None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:21.625670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6cace-ac9d-4bbd-b43a-51d6f73cf38f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:21.626295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6cace-ac9d-4bbd-b43a-51d6f73cf38f service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:21.626574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6cace-ac9d-4bbd-b43a-51d6f73cf38f service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:21.626844 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6cace-ac9d-4bbd-b43a-51d6f73cf38f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:21.628612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-64d6cace-ac9d-4bbd-b43a-51d6f73cf38f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:21.675338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:21.739262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6cace-ac9d-4bbd-b43a-51d6f73cf38f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:21.740798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6cace-ac9d-4bbd-b43a-51d6f73cf38f service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:21.759274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 675/1346] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:26.039906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a4df0-f88d-4cdd-9e8b-aef9a2bf5ab7 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:26.040455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a4df0-f88d-4cdd-9e8b-aef9a2bf5ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:26.040455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a4df0-f88d-4cdd-9e8b-aef9a2bf5ab7 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:26.041828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 672/1347] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:26.048004 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa303b61-da4d-448e-a55a-cfb4766d17bc None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:26.048535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa303b61-da4d-448e-a55a-cfb4766d17bc None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:26.048629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa303b61-da4d-448e-a55a-cfb4766d17bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:26.121792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-aa303b61-da4d-448e-a55a-cfb4766d17bc None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:26.123022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-aa303b61-da4d-448e-a55a-cfb4766d17bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:26.224351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:26.226927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 676/1348] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:26.514652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a4df0-f88d-4cdd-9e8b-aef9a2bf5ab7 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:26.522395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cc7a4df0-f88d-4cdd-9e8b-aef9a2bf5ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:26.628668 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a4df0-f88d-4cdd-9e8b-aef9a2bf5ab7 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:26.678976 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:26.705451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29a2bc-ebec-45c6-8359-84c3a9472e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:26.706054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a2bc-ebec-45c6-8359-84c3a9472e38 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:26.706337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a2bc-ebec-45c6-8359-84c3a9472e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:26.706607 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a2bc-ebec-45c6-8359-84c3a9472e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:26.708236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3b29a2bc-ebec-45c6-8359-84c3a9472e38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:26.815794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29a2bc-ebec-45c6-8359-84c3a9472e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:26.817264 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29a2bc-ebec-45c6-8359-84c3a9472e38 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:26.834660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 673/1349] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:19:31.830728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26111fd-3c96-40b8-8727-c41d7b1dc7b4 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:31.830728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26111fd-3c96-40b8-8727-c41d7b1dc7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:31.830728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26111fd-3c96-40b8-8727-c41d7b1dc7b4 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:31.834158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 677/1350] 10.222.0.85 () {54 vars in 921 bytes} [Mon Aug 29 20:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:31.837737 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b90c38-337c-4272-a2c3-884448062319 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:31.838163 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b90c38-337c-4272-a2c3-884448062319 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:31.838291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b90c38-337c-4272-a2c3-884448062319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:31.954886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-98b90c38-337c-4272-a2c3-884448062319 None None] MFA Rules not processed for user `5288af2dbc15438c8b209b0ba2454cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:31.956616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-98b90c38-337c-4272-a2c3-884448062319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:31.979958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:32.112465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 674/1351] 10.222.0.85 () {58 vars in 1018 bytes} [Mon Aug 29 20:19:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:19:36.463482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d9b17-b2e5-4725-bc61-20b56aa6fc41 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:36.464484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d9b17-b2e5-4725-bc61-20b56aa6fc41 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:36.465487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d9b17-b2e5-4725-bc61-20b56aa6fc41 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:36.467026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 678/1352] 10.222.0.85 () {54 vars in 921 bytes} [Mon Aug 29 20:19:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:36.472047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e567a-0148-4380-a678-9c341b4a707c None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:36.472517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e567a-0148-4380-a678-9c341b4a707c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:36.472717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e567a-0148-4380-a678-9c341b4a707c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:36.556880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b75e567a-0148-4380-a678-9c341b4a707c None None] MFA Rules not processed for user `5288af2dbc15438c8b209b0ba2454cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:36.558082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b75e567a-0148-4380-a678-9c341b4a707c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:36.641140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:36.679152 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 675/1353] 10.222.0.85 () {58 vars in 1018 bytes} [Mon Aug 29 20:19:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:37.863718 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15b106-ce6e-4ae4-8e0e-5ecbe92fad2e None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:37.864256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15b106-ce6e-4ae4-8e0e-5ecbe92fad2e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:37.864256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15b106-ce6e-4ae4-8e0e-5ecbe92fad2e None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:37.865556 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 679/1354] 10.222.0.85 () {54 vars in 921 bytes} [Mon Aug 29 20:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:37.870643 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0506f-2568-429a-b6ad-5b4e5e433d7a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:37.879217 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0506f-2568-429a-b6ad-5b4e5e433d7a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:37.879217 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0506f-2568-429a-b6ad-5b4e5e433d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:37.951749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e4e0506f-2568-429a-b6ad-5b4e5e433d7a None None] MFA Rules not processed for user `5288af2dbc15438c8b209b0ba2454cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:37.953008 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e4e0506f-2568-429a-b6ad-5b4e5e433d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:38.062964 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 676/1355] 10.222.0.85 () {58 vars in 1018 bytes} [Mon Aug 29 20:19:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:41.317011 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd56111-bd4d-48ce-8af6-1cd32849fd1b None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.317538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd56111-bd4d-48ce-8af6-1cd32849fd1b None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.317647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd56111-bd4d-48ce-8af6-1cd32849fd1b None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.318947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 680/1356] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:41.324647 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac87cfb-ab3b-4429-94e6-d02cbc8ee134 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.325075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac87cfb-ab3b-4429-94e6-d02cbc8ee134 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.325265 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac87cfb-ab3b-4429-94e6-d02cbc8ee134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.364542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:41.401982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-0ac87cfb-ab3b-4429-94e6-d02cbc8ee134 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:41.403912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-0ac87cfb-ab3b-4429-94e6-d02cbc8ee134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:41.530609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 677/1357] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:41.847826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f40e3-976f-4dfc-9d40-50e381b4ff9e None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.848463 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f40e3-976f-4dfc-9d40-50e381b4ff9e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.848948 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f40e3-976f-4dfc-9d40-50e381b4ff9e None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.851110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 681/1358] 10.222.0.85 () {54 vars in 962 bytes} [Mon Aug 29 20:19:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:41.859029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788327f8-2739-490d-96c4-5f90f0b2d918 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.859631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788327f8-2739-490d-96c4-5f90f0b2d918 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.860280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788327f8-2739-490d-96c4-5f90f0b2d918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.945511 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-788327f8-2739-490d-96c4-5f90f0b2d918 None None] MFA Rules not processed for user `805df3047fee45c7a702d72a5c082eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:41.946706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-788327f8-2739-490d-96c4-5f90f0b2d918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:41.999302 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:42.067829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 678/1359] 10.222.0.85 () {58 vars in 1059 bytes} [Mon Aug 29 20:19:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:19:42.075822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f40e3-976f-4dfc-9d40-50e381b4ff9e None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:42.077349 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-dd3f40e3-976f-4dfc-9d40-50e381b4ff9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:42.229646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f40e3-976f-4dfc-9d40-50e381b4ff9e None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:42.310277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:42.357591 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee3e1f-7fea-4d77-8ac0-14ad1dca9ba4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:42.358410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee3e1f-7fea-4d77-8ac0-14ad1dca9ba4 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:42.358793 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee3e1f-7fea-4d77-8ac0-14ad1dca9ba4 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:42.359168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee3e1f-7fea-4d77-8ac0-14ad1dca9ba4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:42.361239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-dfee3e1f-7fea-4d77-8ac0-14ad1dca9ba4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:42.456024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee3e1f-7fea-4d77-8ac0-14ad1dca9ba4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:42.457732 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee3e1f-7fea-4d77-8ac0-14ad1dca9ba4 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:42.476880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 682/1360] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:44.604664 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe4bf6-7119-4912-aa0c-840c311b23cb None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:44.604664 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe4bf6-7119-4912-aa0c-840c311b23cb None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:44.604664 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe4bf6-7119-4912-aa0c-840c311b23cb None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:44.606982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 679/1361] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:44.613568 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ecc9d-527d-41b0-aa6b-4e7ec662abaa None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:44.614005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ecc9d-527d-41b0-aa6b-4e7ec662abaa None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:44.614097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ecc9d-527d-41b0-aa6b-4e7ec662abaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:44.690346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-068ecc9d-527d-41b0-aa6b-4e7ec662abaa None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:44.692205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-068ecc9d-527d-41b0-aa6b-4e7ec662abaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:44.733343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:44.795068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 683/1362] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:19:45.088514 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe4bf6-7119-4912-aa0c-840c311b23cb None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:45.090633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5dfe4bf6-7119-4912-aa0c-840c311b23cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:45.268354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe4bf6-7119-4912-aa0c-840c311b23cb None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:45.278682 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:45.387542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90fc6b-c3f2-41c7-860f-87ed2a622746 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:45.388057 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90fc6b-c3f2-41c7-860f-87ed2a622746 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:45.388396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90fc6b-c3f2-41c7-860f-87ed2a622746 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:45.388627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90fc6b-c3f2-41c7-860f-87ed2a622746 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:45.390239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cd90fc6b-c3f2-41c7-860f-87ed2a622746 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:45.484795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90fc6b-c3f2-41c7-860f-87ed2a622746 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:45.487620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90fc6b-c3f2-41c7-860f-87ed2a622746 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:45.508358 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 680/1363] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:47.961870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d81d5-e180-4f06-9074-4002b1bd1fa2 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.962520 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d81d5-e180-4f06-9074-4002b1bd1fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.962520 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d81d5-e180-4f06-9074-4002b1bd1fa2 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.963862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 684/1364] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:47.969885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029dc90-b09e-4608-80ce-7a7a10b2c634 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.970685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029dc90-b09e-4608-80ce-7a7a10b2c634 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.971130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029dc90-b09e-4608-80ce-7a7a10b2c634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.987779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:48.043694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-9029dc90-b09e-4608-80ce-7a7a10b2c634 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:48.045105 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9029dc90-b09e-4608-80ce-7a7a10b2c634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:48.154893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 681/1365] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:48.456555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c077a3-8766-4988-b327-8798e5e49ec2 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:48.456832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c077a3-8766-4988-b327-8798e5e49ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:48.457100 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c077a3-8766-4988-b327-8798e5e49ec2 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:48.458667 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 685/1366] 10.222.0.85 () {54 vars in 962 bytes} [Mon Aug 29 20:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:48.464401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f8965-e974-45bd-8d69-731380e727f0 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:48.464694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f8965-e974-45bd-8d69-731380e727f0 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:48.464969 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f8965-e974-45bd-8d69-731380e727f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:48.546386 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-402f8965-e974-45bd-8d69-731380e727f0 None None] MFA Rules not processed for user `805df3047fee45c7a702d72a5c082eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:48.547508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-402f8965-e974-45bd-8d69-731380e727f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:48.585885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:48.676324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 682/1367] 10.222.0.85 () {58 vars in 1059 bytes} [Mon Aug 29 20:19:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:48.683398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c077a3-8766-4988-b327-8798e5e49ec2 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:48.685427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c4c077a3-8766-4988-b327-8798e5e49ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:48.841855 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c077a3-8766-4988-b327-8798e5e49ec2 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:48.863367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:48.957277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e039c-ec7c-4436-8182-b99c9a556c55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:48.957886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e039c-ec7c-4436-8182-b99c9a556c55 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:48.958158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e039c-ec7c-4436-8182-b99c9a556c55 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:48.958429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e039c-ec7c-4436-8182-b99c9a556c55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:48.960021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-021e039c-ec7c-4436-8182-b99c9a556c55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:49.061080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e039c-ec7c-4436-8182-b99c9a556c55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:49.062708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e039c-ec7c-4436-8182-b99c9a556c55 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:49.081760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 686/1368] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:19:49.555299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c13e4c-8018-4ac8-b73d-794041616d9c None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:49.555816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c13e4c-8018-4ac8-b73d-794041616d9c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:49.555816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c13e4c-8018-4ac8-b73d-794041616d9c None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:49.557311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 683/1369] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:49.562939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38c946-2a89-43f0-aa1c-95d46bc0035c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:49.563229 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38c946-2a89-43f0-aa1c-95d46bc0035c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:49.563501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38c946-2a89-43f0-aa1c-95d46bc0035c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:49.595976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:49.633691 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-1f38c946-2a89-43f0-aa1c-95d46bc0035c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:49.634943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1f38c946-2a89-43f0-aa1c-95d46bc0035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:49.736472 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 687/1370] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:50.332378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe267b-1da5-4d8d-a3c8-5c8acdc66bc8 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:50.332664 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe267b-1da5-4d8d-a3c8-5c8acdc66bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:50.332938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe267b-1da5-4d8d-a3c8-5c8acdc66bc8 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:50.334854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 684/1371] 10.222.0.85 () {54 vars in 964 bytes} [Mon Aug 29 20:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:50.339801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4768a4-b0a1-456e-a729-c96d58072e70 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:50.340346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4768a4-b0a1-456e-a729-c96d58072e70 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:50.340346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4768a4-b0a1-456e-a729-c96d58072e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:50.419211 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-cd4768a4-b0a1-456e-a729-c96d58072e70 None None] MFA Rules not processed for user `9d90a7ed44fd4a1fa94b9d44939df425`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:50.420685 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-cd4768a4-b0a1-456e-a729-c96d58072e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:50.478474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:50.509590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 688/1372] 10.222.0.85 () {58 vars in 1061 bytes} [Mon Aug 29 20:19:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:50.518024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe267b-1da5-4d8d-a3c8-5c8acdc66bc8 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:50.519809 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-64fe267b-1da5-4d8d-a3c8-5c8acdc66bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:50.628901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe267b-1da5-4d8d-a3c8-5c8acdc66bc8 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:50.669978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:50.707663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb32d21-9bdb-4910-b9c7-209076a967c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:50.708984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb32d21-9bdb-4910-b9c7-209076a967c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:50.709106 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb32d21-9bdb-4910-b9c7-209076a967c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:50.709313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb32d21-9bdb-4910-b9c7-209076a967c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:50.711412 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ceb32d21-9bdb-4910-b9c7-209076a967c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:50.823430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb32d21-9bdb-4910-b9c7-209076a967c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:50.825603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb32d21-9bdb-4910-b9c7-209076a967c7 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:50.857350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 685/1373] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:51.289285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550e438-a1fc-4608-89e2-f15aaf44b130 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:51.289679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550e438-a1fc-4608-89e2-f15aaf44b130 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:51.290042 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550e438-a1fc-4608-89e2-f15aaf44b130 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:51.291528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 689/1374] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:51.297215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1875cb-047b-455d-9045-dd872e9a6a02 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:51.297502 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1875cb-047b-455d-9045-dd872e9a6a02 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:51.297787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1875cb-047b-455d-9045-dd872e9a6a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:51.363929 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-fc1875cb-047b-455d-9045-dd872e9a6a02 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:51.365156 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-fc1875cb-047b-455d-9045-dd872e9a6a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:51.385697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:51.467700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 686/1375] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:51.759425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550e438-a1fc-4608-89e2-f15aaf44b130 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:51.761638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f550e438-a1fc-4608-89e2-f15aaf44b130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:51.908527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550e438-a1fc-4608-89e2-f15aaf44b130 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:51.960949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:52.022670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdd1b0-2ba2-48d1-bc5d-751e33c4f7f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:52.023195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdd1b0-2ba2-48d1-bc5d-751e33c4f7f5 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:52.023481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdd1b0-2ba2-48d1-bc5d-751e33c4f7f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:52.023757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdd1b0-2ba2-48d1-bc5d-751e33c4f7f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:52.025406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a2fdd1b0-2ba2-48d1-bc5d-751e33c4f7f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:52.114382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdd1b0-2ba2-48d1-bc5d-751e33c4f7f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:52.122721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdd1b0-2ba2-48d1-bc5d-751e33c4f7f5 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:52.134763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 690/1376] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:54.232273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c27f7-049e-4bf1-9175-aed4a2252c35 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:54.232819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c27f7-049e-4bf1-9175-aed4a2252c35 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:54.232819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c27f7-049e-4bf1-9175-aed4a2252c35 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:54.234224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 687/1377] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:54.239429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5195b8-2f82-4802-987b-221e67755451 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:54.239821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5195b8-2f82-4802-987b-221e67755451 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:54.239913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5195b8-2f82-4802-987b-221e67755451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:54.307024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-8a5195b8-2f82-4802-987b-221e67755451 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:54.308289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8a5195b8-2f82-4802-987b-221e67755451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:54.315363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:54.407059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 691/1378] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:54.709508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c27f7-049e-4bf1-9175-aed4a2252c35 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:54.710968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-495c27f7-049e-4bf1-9175-aed4a2252c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:54.874095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c27f7-049e-4bf1-9175-aed4a2252c35 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:54.877987 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:55.004909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f46b99-0420-4691-b6e2-b00c136806ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:55.005522 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f46b99-0420-4691-b6e2-b00c136806ae service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:55.005803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f46b99-0420-4691-b6e2-b00c136806ae service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:55.006075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f46b99-0420-4691-b6e2-b00c136806ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:55.007948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e9f46b99-0420-4691-b6e2-b00c136806ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:55.107595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f46b99-0420-4691-b6e2-b00c136806ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:55.109660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f46b99-0420-4691-b6e2-b00c136806ae service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:55.131406 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 688/1379] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:19:57.278899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2fef4-2a34-40e6-838b-3e759ddc6c67 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.279502 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2fef4-2a34-40e6-838b-3e759ddc6c67 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.279502 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2fef4-2a34-40e6-838b-3e759ddc6c67 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.280917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 692/1380] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:57.285951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abc257-45f3-43da-a8e9-74ce9d3708df None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.286381 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abc257-45f3-43da-a8e9-74ce9d3708df None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.286484 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abc257-45f3-43da-a8e9-74ce9d3708df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.299211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:57.357373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-f7abc257-45f3-43da-a8e9-74ce9d3708df None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:57.358839 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-f7abc257-45f3-43da-a8e9-74ce9d3708df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:57.460276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 689/1381] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:19:57.750087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2fef4-2a34-40e6-838b-3e759ddc6c67 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:57.751598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-d1b2fef4-2a34-40e6-838b-3e759ddc6c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:57.901192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:57.912370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2fef4-2a34-40e6-838b-3e759ddc6c67 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:58.030001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41ed30-84bd-4f75-b649-8cb754d2d838 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:58.030605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41ed30-84bd-4f75-b649-8cb754d2d838 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:58.030884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41ed30-84bd-4f75-b649-8cb754d2d838 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:58.031155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41ed30-84bd-4f75-b649-8cb754d2d838 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:58.032828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-dc41ed30-84bd-4f75-b649-8cb754d2d838 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:58.131053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41ed30-84bd-4f75-b649-8cb754d2d838 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:58.132888 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41ed30-84bd-4f75-b649-8cb754d2d838 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:58.145004 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:58.153732 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 693/1382] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:00.222295 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931eb50-ed6e-473c-8b4d-81db3b2d0c31 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:00.222992 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931eb50-ed6e-473c-8b4d-81db3b2d0c31 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:00.222992 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931eb50-ed6e-473c-8b4d-81db3b2d0c31 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:00.224227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 690/1383] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:00.230286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33888ba6-ef56-483d-aa65-9a110757601c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:00.230843 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33888ba6-ef56-483d-aa65-9a110757601c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:00.230843 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33888ba6-ef56-483d-aa65-9a110757601c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:00.302238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-33888ba6-ef56-483d-aa65-9a110757601c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:00.303455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-33888ba6-ef56-483d-aa65-9a110757601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:00.401012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 694/1384] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:00.696012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931eb50-ed6e-473c-8b4d-81db3b2d0c31 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:00.697565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2931eb50-ed6e-473c-8b4d-81db3b2d0c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:00.788507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:00.853328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931eb50-ed6e-473c-8b4d-81db3b2d0c31 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:00.983634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ab610-ae9b-495a-a2a2-3d7dd77c674b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:00.984272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ab610-ae9b-495a-a2a2-3d7dd77c674b service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:00.984558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ab610-ae9b-495a-a2a2-3d7dd77c674b service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:00.984833 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ab610-ae9b-495a-a2a2-3d7dd77c674b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:00.986525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-f28ab610-ae9b-495a-a2a2-3d7dd77c674b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:01.037392 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:01.076997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ab610-ae9b-495a-a2a2-3d7dd77c674b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:01.078643 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ab610-ae9b-495a-a2a2-3d7dd77c674b service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:01.097599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 691/1385] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:03.048742 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006d7da-28a8-456c-9506-2d37b5d1dcf3 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:03.049285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006d7da-28a8-456c-9506-2d37b5d1dcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:03.049452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006d7da-28a8-456c-9506-2d37b5d1dcf3 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:03.050952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 695/1386] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:20:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:03.056043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732cd98-8d97-4593-967e-72a35da542df None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:03.056528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732cd98-8d97-4593-967e-72a35da542df None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:03.056625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732cd98-8d97-4593-967e-72a35da542df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:03.127825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-7732cd98-8d97-4593-967e-72a35da542df None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:03.129109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7732cd98-8d97-4593-967e-72a35da542df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:03.228881 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 692/1387] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:03.564508 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006d7da-28a8-456c-9506-2d37b5d1dcf3 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:03.566305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-d006d7da-28a8-456c-9506-2d37b5d1dcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:03.649887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:03.720306 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006d7da-28a8-456c-9506-2d37b5d1dcf3 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:03.835420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da49a1-ff5e-4327-9a4d-46bf713613b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:03.835944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da49a1-ff5e-4327-9a4d-46bf713613b5 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:03.836250 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da49a1-ff5e-4327-9a4d-46bf713613b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:03.836524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da49a1-ff5e-4327-9a4d-46bf713613b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:03.838112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-65da49a1-ff5e-4327-9a4d-46bf713613b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:03.880342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:03.940908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da49a1-ff5e-4327-9a4d-46bf713613b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:03.942499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da49a1-ff5e-4327-9a4d-46bf713613b5 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:03.961274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 696/1388] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:06.144650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043a045-1283-4ce4-9dcf-4b8bd6385d3d None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.145193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043a045-1283-4ce4-9dcf-4b8bd6385d3d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.145193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043a045-1283-4ce4-9dcf-4b8bd6385d3d None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.146526 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 693/1389] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:06.151940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55279d0d-3dc8-4254-97e2-215b786eabfe None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.153073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55279d0d-3dc8-4254-97e2-215b786eabfe None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.153073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55279d0d-3dc8-4254-97e2-215b786eabfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.221172 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-55279d0d-3dc8-4254-97e2-215b786eabfe None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:06.222509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-55279d0d-3dc8-4254-97e2-215b786eabfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:06.320451 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 697/1390] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:06.630884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043a045-1283-4ce4-9dcf-4b8bd6385d3d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:06.632919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e043a045-1283-4ce4-9dcf-4b8bd6385d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:06.671916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:06.792109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043a045-1283-4ce4-9dcf-4b8bd6385d3d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:06.896636 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:06.908885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b309f0-048e-4578-968a-1f2734942662 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:06.909488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b309f0-048e-4578-968a-1f2734942662 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.909769 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b309f0-048e-4578-968a-1f2734942662 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.910037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b309f0-048e-4578-968a-1f2734942662 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.911653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-42b309f0-048e-4578-968a-1f2734942662 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:07.008442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b309f0-048e-4578-968a-1f2734942662 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:07.011136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b309f0-048e-4578-968a-1f2734942662 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:07.034969 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 694/1391] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:09.228050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758e322-9d7a-48bf-baa7-55c28c97360b None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:09.228658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758e322-9d7a-48bf-baa7-55c28c97360b None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:09.228658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758e322-9d7a-48bf-baa7-55c28c97360b None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:09.230019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 698/1392] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:09.235239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9bc25-634a-42ef-abd3-46ced4cf499f None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:09.235648 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9bc25-634a-42ef-abd3-46ced4cf499f None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:09.235813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9bc25-634a-42ef-abd3-46ced4cf499f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:09.302311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-9ba9bc25-634a-42ef-abd3-46ced4cf499f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:09.303569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9ba9bc25-634a-42ef-abd3-46ced4cf499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:09.363146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:09.402652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 695/1393] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:09.692289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758e322-9d7a-48bf-baa7-55c28c97360b None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:09.693808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2758e322-9d7a-48bf-baa7-55c28c97360b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:09.707421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:09.842416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758e322-9d7a-48bf-baa7-55c28c97360b None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:09.929525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:09.956263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a2dc4-92bf-49b1-9bd7-d0db20250d10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:09.956872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a2dc4-92bf-49b1-9bd7-d0db20250d10 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:09.957147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a2dc4-92bf-49b1-9bd7-d0db20250d10 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:09.957417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a2dc4-92bf-49b1-9bd7-d0db20250d10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:09.959062 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2a9a2dc4-92bf-49b1-9bd7-d0db20250d10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:10.050735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a2dc4-92bf-49b1-9bd7-d0db20250d10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:10.052411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a2dc4-92bf-49b1-9bd7-d0db20250d10 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:10.071808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 699/1394] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:10.498623 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16349a57-eca5-4ea4-a175-d5da0b425743 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:10.499200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16349a57-eca5-4ea4-a175-d5da0b425743 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:10.499200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16349a57-eca5-4ea4-a175-d5da0b425743 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:10.501321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 696/1395] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:20:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:10.507352 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365803db-cc3d-4e85-a31a-f22bb6856b2c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:10.508057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365803db-cc3d-4e85-a31a-f22bb6856b2c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:10.508057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365803db-cc3d-4e85-a31a-f22bb6856b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:10.580254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-365803db-cc3d-4e85-a31a-f22bb6856b2c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:10.581626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-365803db-cc3d-4e85-a31a-f22bb6856b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:10.633268 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:10.682413 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 700/1396] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:10.982782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5d3db-b434-4636-9c15-8672db86a6de None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:10.983160 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5d3db-b434-4636-9c15-8672db86a6de None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:10.983504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5d3db-b434-4636-9c15-8672db86a6de None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:10.984989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 697/1397] 10.222.0.85 () {54 vars in 967 bytes} [Mon Aug 29 20:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:10.990011 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe46220-0ac4-4b25-8d42-8962e4e9a2c2 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:10.990299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe46220-0ac4-4b25-8d42-8962e4e9a2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:10.990577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe46220-0ac4-4b25-8d42-8962e4e9a2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:11.066261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-ebe46220-0ac4-4b25-8d42-8962e4e9a2c2 None None] MFA Rules not processed for user `a7e768fdd1054e40b6b4910e39a6e533`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:11.067500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ebe46220-0ac4-4b25-8d42-8962e4e9a2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:11.144404 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 701/1398] 10.222.0.85 () {58 vars in 1064 bytes} [Mon Aug 29 20:20:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:11.151554 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5d3db-b434-4636-9c15-8672db86a6de None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:11.153098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3fe5d3db-b434-4636-9c15-8672db86a6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:11.257518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5d3db-b434-4636-9c15-8672db86a6de None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:11.331633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0555e-f3d5-4e9d-9c60-139c79e4fe09 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7e768fdd1054e40b6b4910e39a6e533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:11.332391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0555e-f3d5-4e9d-9c60-139c79e4fe09 service glance] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:11.332689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0555e-f3d5-4e9d-9c60-139c79e4fe09 service glance] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:11.332961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0555e-f3d5-4e9d-9c60-139c79e4fe09 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:11.334581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-29e0555e-f3d5-4e9d-9c60-139c79e4fe09 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:11.352974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:11.444561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0555e-f3d5-4e9d-9c60-139c79e4fe09 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:11.446039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0555e-f3d5-4e9d-9c60-139c79e4fe09 service glance] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:11.466279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 698/1399] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:12.303785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466e48-401e-49fb-af26-d21b0d8590ad None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.304264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466e48-401e-49fb-af26-d21b0d8590ad None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.304371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466e48-401e-49fb-af26-d21b0d8590ad None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.305726 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 702/1400] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:12.311619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13033b66-5300-432a-84fe-7ffc12930316 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.312047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13033b66-5300-432a-84fe-7ffc12930316 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.312188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13033b66-5300-432a-84fe-7ffc12930316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.379012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-13033b66-5300-432a-84fe-7ffc12930316 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:12.380283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-13033b66-5300-432a-84fe-7ffc12930316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:12.447595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:12.479488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 699/1401] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:12.517466 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc9336-1e05-421f-9fc3-1b3169c1dea3 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.517756 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc9336-1e05-421f-9fc3-1b3169c1dea3 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.518038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc9336-1e05-421f-9fc3-1b3169c1dea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.578991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:12.587203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7bdc9336-1e05-421f-9fc3-1b3169c1dea3 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:12.588983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7bdc9336-1e05-421f-9fc3-1b3169c1dea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:12.679787 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 703/1402] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:20:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:12.722811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70192e66-5c6b-4442-98da-10a1dbee31c3 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.723098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70192e66-5c6b-4442-98da-10a1dbee31c3 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.723371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70192e66-5c6b-4442-98da-10a1dbee31c3 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.724842 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 700/1403] 10.222.0.85 () {54 vars in 943 bytes} [Mon Aug 29 20:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:12.731754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978a3ac-a1ed-4beb-b3bc-f87377c78f9b None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.732047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978a3ac-a1ed-4beb-b3bc-f87377c78f9b None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.732374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978a3ac-a1ed-4beb-b3bc-f87377c78f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.776961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70192e66-5c6b-4442-98da-10a1dbee31c3 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:12.778467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-70192e66-5c6b-4442-98da-10a1dbee31c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:12.810840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-6978a3ac-a1ed-4beb-b3bc-f87377c78f9b None None] MFA Rules not processed for user `d52f84245a2142e69c309aa709a7d921`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:12.812042 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-6978a3ac-a1ed-4beb-b3bc-f87377c78f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:12.887146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:12.923772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 704/1404] 10.222.0.85 () {58 vars in 1040 bytes} [Mon Aug 29 20:20:12 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:12.930928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70192e66-5c6b-4442-98da-10a1dbee31c3 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:12.933122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978a3ac-a1ed-4beb-b3bc-f87377c78f9b None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:12.944454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978a3ac-a1ed-4beb-b3bc-f87377c78f9b None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:12.989388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:13.048252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618118d1-e5b2-49e4-952a-3566d8fc859a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:13.048851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618118d1-e5b2-49e4-952a-3566d8fc859a service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.049124 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618118d1-e5b2-49e4-952a-3566d8fc859a service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.049391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618118d1-e5b2-49e4-952a-3566d8fc859a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.051111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-618118d1-e5b2-49e4-952a-3566d8fc859a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:13.052907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180e291-8387-4df8-a0a9-f6e25999d67f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:13.053512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180e291-8387-4df8-a0a9-f6e25999d67f service swift] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.053790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180e291-8387-4df8-a0a9-f6e25999d67f service swift] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.054059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180e291-8387-4df8-a0a9-f6e25999d67f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.055648 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4180e291-8387-4df8-a0a9-f6e25999d67f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:13.137459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:13.139753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180e291-8387-4df8-a0a9-f6e25999d67f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:13.141380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180e291-8387-4df8-a0a9-f6e25999d67f service swift] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:13.143890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618118d1-e5b2-49e4-952a-3566d8fc859a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:13.145669 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618118d1-e5b2-49e4-952a-3566d8fc859a service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:13.149623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 705/1405] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:20:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 20:20:13.167424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 701/1406] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:20:15.299968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a511c6-b997-4461-84d8-b6aa99ba13e3 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:15.300688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a511c6-b997-4461-84d8-b6aa99ba13e3 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:15.300688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a511c6-b997-4461-84d8-b6aa99ba13e3 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:15.302002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 706/1407] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:20:15.307717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9119e13-721b-4e78-b5e8-501dad5529e6 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:15.308647 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9119e13-721b-4e78-b5e8-501dad5529e6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:15.309172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9119e13-721b-4e78-b5e8-501dad5529e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:15.385531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e9119e13-721b-4e78-b5e8-501dad5529e6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:15.386920 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e9119e13-721b-4e78-b5e8-501dad5529e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:15.402341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:15.496826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 702/1408] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:15.799227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a511c6-b997-4461-84d8-b6aa99ba13e3 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:15.800842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-00a511c6-b997-4461-84d8-b6aa99ba13e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:15.966883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a511c6-b997-4461-84d8-b6aa99ba13e3 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:16.043456 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:16.095413 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b99de-7b0a-4cbb-afc9-3d79b1584570 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:16.098022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b99de-7b0a-4cbb-afc9-3d79b1584570 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:16.098022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b99de-7b0a-4cbb-afc9-3d79b1584570 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:16.098022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b99de-7b0a-4cbb-afc9-3d79b1584570 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:16.099921 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4a3b99de-7b0a-4cbb-afc9-3d79b1584570 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:16.196766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b99de-7b0a-4cbb-afc9-3d79b1584570 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:16.198403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b99de-7b0a-4cbb-afc9-3d79b1584570 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:16.219715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 707/1409] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:26.154501 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=38 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:26.447447 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=64 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:37.266023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388a421-e747-4941-bb69-ad80353a7ffe None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:37.266619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388a421-e747-4941-bb69-ad80353a7ffe None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:37.266619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388a421-e747-4941-bb69-ad80353a7ffe None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:37.267903 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 703/1410] 10.222.0.85 () {54 vars in 918 bytes} [Mon Aug 29 20:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:37.273431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d964cf-4e3c-4abc-92fd-8812a9442a95 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:37.273817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d964cf-4e3c-4abc-92fd-8812a9442a95 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:37.273983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d964cf-4e3c-4abc-92fd-8812a9442a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:37.348967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-22d964cf-4e3c-4abc-92fd-8812a9442a95 None None] MFA Rules not processed for user `5288af2dbc15438c8b209b0ba2454cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:37.350174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-22d964cf-4e3c-4abc-92fd-8812a9442a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:37.453506 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 708/1411] 10.222.0.85 () {58 vars in 1015 bytes} [Mon Aug 29 20:20:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:37.467969 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71620867-17f8-4c1e-8454-6fed0856737b None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:37.468284 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71620867-17f8-4c1e-8454-6fed0856737b None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:37.468559 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71620867-17f8-4c1e-8454-6fed0856737b None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:37.469873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 704/1412] 10.222.0.85 () {54 vars in 918 bytes} [Mon Aug 29 20:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:37.474323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f2ae1-a7be-4a40-8283-da2528e00283 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:37.474607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f2ae1-a7be-4a40-8283-da2528e00283 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:37.474878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f2ae1-a7be-4a40-8283-da2528e00283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:37.551156 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:37.554073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-326f2ae1-a7be-4a40-8283-da2528e00283 None None] MFA Rules not processed for user `805df3047fee45c7a702d72a5c082eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:37.555272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-326f2ae1-a7be-4a40-8283-da2528e00283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:37.675085 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 709/1413] 10.222.0.85 () {58 vars in 1015 bytes} [Mon Aug 29 20:20:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:20:40.072397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7cba9-cd0d-431b-acf2-ee831c2946c6 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:40.072998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7cba9-cd0d-431b-acf2-ee831c2946c6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:40.072998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7cba9-cd0d-431b-acf2-ee831c2946c6 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:40.074309 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:40.079661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3e119-4a18-44f0-b515-ff6734e385b5 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:40.080064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3e119-4a18-44f0-b515-ff6734e385b5 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:40.080230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3e119-4a18-44f0-b515-ff6734e385b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:40.145041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-2aa3e119-4a18-44f0-b515-ff6734e385b5 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:40.146174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2aa3e119-4a18-44f0-b515-ff6734e385b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:40.200250 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:40.241651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:40.531305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7cba9-cd0d-431b-acf2-ee831c2946c6 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:40.532911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-82a7cba9-cd0d-431b-acf2-ee831c2946c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:40.681752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7cba9-cd0d-431b-acf2-ee831c2946c6 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:40.753384 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:40.793966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2a154-fc58-4f96-8227-25bbaef4a1cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:40.794492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2a154-fc58-4f96-8227-25bbaef4a1cd service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:40.794774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2a154-fc58-4f96-8227-25bbaef4a1cd service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:40.795039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2a154-fc58-4f96-8227-25bbaef4a1cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:40.797364 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-01f2a154-fc58-4f96-8227-25bbaef4a1cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:40.886777 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2a154-fc58-4f96-8227-25bbaef4a1cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:40.888399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2a154-fc58-4f96-8227-25bbaef4a1cd service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:40.907356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 706/1416] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:41.796101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb16c0-d38d-417f-b85d-fb32e87a3ae6 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:41.796694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb16c0-d38d-417f-b85d-fb32e87a3ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:41.796694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb16c0-d38d-417f-b85d-fb32e87a3ae6 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:41.798076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:41.802844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53becc3a-7f4e-40bc-9a16-862794666d50 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:41.803252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53becc3a-7f4e-40bc-9a16-862794666d50 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:41.803399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53becc3a-7f4e-40bc-9a16-862794666d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:41.877912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-53becc3a-7f4e-40bc-9a16-862794666d50 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:41.879015 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-53becc3a-7f4e-40bc-9a16-862794666d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:41.905913 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:41.975030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:42.124327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb16c0-d38d-417f-b85d-fb32e87a3ae6 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:42.125849 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-badb16c0-d38d-417f-b85d-fb32e87a3ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:42.272531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb16c0-d38d-417f-b85d-fb32e87a3ae6 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:42.315330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:42.388261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b5339-179c-46fd-ba7d-481f99e781aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:42.388866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b5339-179c-46fd-ba7d-481f99e781aa service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:42.389143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b5339-179c-46fd-ba7d-481f99e781aa service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:42.389416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b5339-179c-46fd-ba7d-481f99e781aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:42.391000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1f5b5339-179c-46fd-ba7d-481f99e781aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:42.477990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b5339-179c-46fd-ba7d-481f99e781aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:42.479588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b5339-179c-46fd-ba7d-481f99e781aa service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:42.497951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 712/1419] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:43.380505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cc58c-36c5-4aaa-b351-89a223563114 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:43.381029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cc58c-36c5-4aaa-b351-89a223563114 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:43.381029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cc58c-36c5-4aaa-b351-89a223563114 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:43.382317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:43.387375 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cf200-ec2e-4fcd-9061-116a0a784190 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:43.387801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cf200-ec2e-4fcd-9061-116a0a784190 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:43.387926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cf200-ec2e-4fcd-9061-116a0a784190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:43.442677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:43.464526 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-fb5cf200-ec2e-4fcd-9061-116a0a784190 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:43.465737 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-fb5cf200-ec2e-4fcd-9061-116a0a784190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:43.562154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:43.716713 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cc58c-36c5-4aaa-b351-89a223563114 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:43.718256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-658cc58c-36c5-4aaa-b351-89a223563114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:43.863023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cc58c-36c5-4aaa-b351-89a223563114 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:43.875507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:43.978489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44ea8f-4d12-46d8-a2f9-ce0f14c2164f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:43.979004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44ea8f-4d12-46d8-a2f9-ce0f14c2164f service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:43.979283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44ea8f-4d12-46d8-a2f9-ce0f14c2164f service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:43.979555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44ea8f-4d12-46d8-a2f9-ce0f14c2164f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:43.981203 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ba44ea8f-4d12-46d8-a2f9-ce0f14c2164f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:44.070040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44ea8f-4d12-46d8-a2f9-ce0f14c2164f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:44.072278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44ea8f-4d12-46d8-a2f9-ce0f14c2164f service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:44.089808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 709/1422] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:44.949025 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242667af-ea3b-4650-8cbe-0cccec037666 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:44.949646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242667af-ea3b-4650-8cbe-0cccec037666 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:44.949646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242667af-ea3b-4650-8cbe-0cccec037666 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:44.950966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:44.955822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a3690-4b52-4190-b5d5-269d6adc83b6 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:44.956258 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a3690-4b52-4190-b5d5-269d6adc83b6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:44.956410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a3690-4b52-4190-b5d5-269d6adc83b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:44.978342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:45.034504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b13a3690-4b52-4190-b5d5-269d6adc83b6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:45.035710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b13a3690-4b52-4190-b5d5-269d6adc83b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:45.131871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:45.280854 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242667af-ea3b-4650-8cbe-0cccec037666 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:45.282289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-242667af-ea3b-4650-8cbe-0cccec037666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:45.391508 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:45.427525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242667af-ea3b-4650-8cbe-0cccec037666 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:45.538352 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67818d-0929-4f57-b24b-bf996e9d4e26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:45.538871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67818d-0929-4f57-b24b-bf996e9d4e26 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:45.539149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67818d-0929-4f57-b24b-bf996e9d4e26 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:45.539420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67818d-0929-4f57-b24b-bf996e9d4e26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:45.541046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-db67818d-0929-4f57-b24b-bf996e9d4e26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:45.616190 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:45.629372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67818d-0929-4f57-b24b-bf996e9d4e26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:45.630961 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67818d-0929-4f57-b24b-bf996e9d4e26 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:45.649934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 715/1425] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:46.508296 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b180d33-8723-4c66-91a7-531b2526dcea None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:46.509472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b180d33-8723-4c66-91a7-531b2526dcea None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:46.509472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b180d33-8723-4c66-91a7-531b2526dcea None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:46.510779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:20:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:46.515871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367ff9a-8b37-4607-af66-2041b1016da1 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:46.516323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367ff9a-8b37-4607-af66-2041b1016da1 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:46.516453 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367ff9a-8b37-4607-af66-2041b1016da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:46.592970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-a367ff9a-8b37-4607-af66-2041b1016da1 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:46.594110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a367ff9a-8b37-4607-af66-2041b1016da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:46.690350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:20:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:46.839769 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b180d33-8723-4c66-91a7-531b2526dcea None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:46.841419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-1b180d33-8723-4c66-91a7-531b2526dcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:46.926809 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:46.989760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b180d33-8723-4c66-91a7-531b2526dcea None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:47.104110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06ba41-8553-4fd8-80e3-9493b422af59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:47.104736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06ba41-8553-4fd8-80e3-9493b422af59 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:47.105012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06ba41-8553-4fd8-80e3-9493b422af59 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:47.105277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06ba41-8553-4fd8-80e3-9493b422af59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:47.106870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-de06ba41-8553-4fd8-80e3-9493b422af59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:47.153949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:47.194605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06ba41-8553-4fd8-80e3-9493b422af59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:47.196749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06ba41-8553-4fd8-80e3-9493b422af59 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:47.214128 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 712/1428] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:48.098237 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024af4a8-5963-4de4-ae2a-2b44545de98c None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.098784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024af4a8-5963-4de4-ae2a-2b44545de98c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.098784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024af4a8-5963-4de4-ae2a-2b44545de98c None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.100123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:48.105620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae6d0-b793-45c1-b294-7a6042a31c5e None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.106038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae6d0-b793-45c1-b294-7a6042a31c5e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.106243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae6d0-b793-45c1-b294-7a6042a31c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.182257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d90ae6d0-b793-45c1-b294-7a6042a31c5e None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:48.183455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d90ae6d0-b793-45c1-b294-7a6042a31c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:48.277930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:48.429331 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024af4a8-5963-4de4-ae2a-2b44545de98c None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:48.431416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-024af4a8-5963-4de4-ae2a-2b44545de98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:48.478952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:48.577033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024af4a8-5963-4de4-ae2a-2b44545de98c None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:48.704919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6fa5f-7abe-4078-ad61-d4b4ea51cc52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:48.705483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6fa5f-7abe-4078-ad61-d4b4ea51cc52 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.705762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6fa5f-7abe-4078-ad61-d4b4ea51cc52 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.706037 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6fa5f-7abe-4078-ad61-d4b4ea51cc52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.708754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-bdc6fa5f-7abe-4078-ad61-d4b4ea51cc52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:48.716534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:48.800898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6fa5f-7abe-4078-ad61-d4b4ea51cc52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:48.802490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6fa5f-7abe-4078-ad61-d4b4ea51cc52 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:48.820515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 718/1431] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:49.693675 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf90624-bdbd-45c6-a3c7-a9f05721be4c None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.694263 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf90624-bdbd-45c6-a3c7-a9f05721be4c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.694263 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf90624-bdbd-45c6-a3c7-a9f05721be4c None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.695559 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:49.700579 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbbd46-a547-4288-abea-c5e1377b71b6 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.701028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbbd46-a547-4288-abea-c5e1377b71b6 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.701138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbbd46-a547-4288-abea-c5e1377b71b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.776188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-29bbbd46-a547-4288-abea-c5e1377b71b6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:49.777411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-29bbbd46-a547-4288-abea-c5e1377b71b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:49.843825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:49.872576 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:50.025970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf90624-bdbd-45c6-a3c7-a9f05721be4c None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:50.027488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ecf90624-bdbd-45c6-a3c7-a9f05721be4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:50.045061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:50.174727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf90624-bdbd-45c6-a3c7-a9f05721be4c None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:50.262274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:50.288388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a277bc-cbef-47ba-9df3-915a4d5ff863 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:50.288982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a277bc-cbef-47ba-9df3-915a4d5ff863 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:50.289260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a277bc-cbef-47ba-9df3-915a4d5ff863 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:50.290028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a277bc-cbef-47ba-9df3-915a4d5ff863 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:50.291616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-86a277bc-cbef-47ba-9df3-915a4d5ff863 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:50.380079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a277bc-cbef-47ba-9df3-915a4d5ff863 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:50.381704 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a277bc-cbef-47ba-9df3-915a4d5ff863 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:50.400332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 715/1434] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:51.299177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda23867-1f6c-4701-a300-f3683a6ac104 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:51.299754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda23867-1f6c-4701-a300-f3683a6ac104 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:51.299754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda23867-1f6c-4701-a300-f3683a6ac104 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:51.301104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:51.306549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fff3b-9a19-45e6-8434-7b6a8668c53a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:51.306988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fff3b-9a19-45e6-8434-7b6a8668c53a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:51.307121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fff3b-9a19-45e6-8434-7b6a8668c53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:51.381030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e49fff3b-9a19-45e6-8434-7b6a8668c53a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:51.382205 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e49fff3b-9a19-45e6-8434-7b6a8668c53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:51.423274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:51.481271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:51.629900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda23867-1f6c-4701-a300-f3683a6ac104 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:51.631424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-dda23867-1f6c-4701-a300-f3683a6ac104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:51.774493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda23867-1f6c-4701-a300-f3683a6ac104 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:51.827987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:51.886582 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fc80e-9aa7-4f9d-a5d7-f7bc5aa6d847 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:51.887104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fc80e-9aa7-4f9d-a5d7-f7bc5aa6d847 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:51.887379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fc80e-9aa7-4f9d-a5d7-f7bc5aa6d847 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:51.887647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fc80e-9aa7-4f9d-a5d7-f7bc5aa6d847 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:51.889268 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-377fc80e-9aa7-4f9d-a5d7-f7bc5aa6d847 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:51.978495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fc80e-9aa7-4f9d-a5d7-f7bc5aa6d847 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:51.980086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fc80e-9aa7-4f9d-a5d7-f7bc5aa6d847 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:51.997625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 721/1437] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:52.879929 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00334bc2-7668-47cf-8f1e-725312541f2a None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:52.880539 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00334bc2-7668-47cf-8f1e-725312541f2a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:52.880539 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00334bc2-7668-47cf-8f1e-725312541f2a None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:52.881823 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:52.887553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce381de-aaee-48a8-88a5-1c70be211e0e None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:52.887997 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce381de-aaee-48a8-88a5-1c70be211e0e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:52.888098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce381de-aaee-48a8-88a5-1c70be211e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:52.964242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-0ce381de-aaee-48a8-88a5-1c70be211e0e None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:52.965565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0ce381de-aaee-48a8-88a5-1c70be211e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:52.967583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:53.061526 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:53.213585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00334bc2-7668-47cf-8f1e-725312541f2a None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:53.215102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-00334bc2-7668-47cf-8f1e-725312541f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:53.358359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00334bc2-7668-47cf-8f1e-725312541f2a None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:53.386167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:53.469578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2faf9-21d5-4532-b5cd-83eb7523a38f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:53.470125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2faf9-21d5-4532-b5cd-83eb7523a38f service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:53.470421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2faf9-21d5-4532-b5cd-83eb7523a38f service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:53.470686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2faf9-21d5-4532-b5cd-83eb7523a38f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:53.472311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-6ac2faf9-21d5-4532-b5cd-83eb7523a38f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:53.561804 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2faf9-21d5-4532-b5cd-83eb7523a38f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:53.563397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2faf9-21d5-4532-b5cd-83eb7523a38f service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:53.581186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 718/1440] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:55.393124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7770ea-6c98-4c85-ba6c-9e5d1ae7f7a3 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:55.393981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7770ea-6c98-4c85-ba6c-9e5d1ae7f7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:55.393981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7770ea-6c98-4c85-ba6c-9e5d1ae7f7a3 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:55.395281 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:55.401092 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdff15-2e19-44c5-8d01-1f39c244e7b1 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:55.401523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdff15-2e19-44c5-8d01-1f39c244e7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:55.401637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdff15-2e19-44c5-8d01-1f39c244e7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:55.441706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:55.466482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-2bfdff15-2e19-44c5-8d01-1f39c244e7b1 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:55.467689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2bfdff15-2e19-44c5-8d01-1f39c244e7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:55.563328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:55.850473 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7770ea-6c98-4c85-ba6c-9e5d1ae7f7a3 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:55.852046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-de7770ea-6c98-4c85-ba6c-9e5d1ae7f7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:55.954249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7770ea-6c98-4c85-ba6c-9e5d1ae7f7a3 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:55.985549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:56.029069 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6c51e-e933-47dd-b31d-34a4be4266ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:56.029678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6c51e-e933-47dd-b31d-34a4be4266ad service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:56.029956 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6c51e-e933-47dd-b31d-34a4be4266ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:56.030227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6c51e-e933-47dd-b31d-34a4be4266ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:56.032001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-97b6c51e-e933-47dd-b31d-34a4be4266ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:56.149842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6c51e-e933-47dd-b31d-34a4be4266ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:56.151211 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6c51e-e933-47dd-b31d-34a4be4266ad service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:56.171087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 724/1443] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:20:58.225510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11d984-fcdf-4411-af60-51d484a2a4a8 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:58.226182 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11d984-fcdf-4411-af60-51d484a2a4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:58.226182 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11d984-fcdf-4411-af60-51d484a2a4a8 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:58.228053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:20:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:58.233242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da81a73-2c19-42d2-ae43-cdf6c54451b6 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:58.233674 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da81a73-2c19-42d2-ae43-cdf6c54451b6 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:58.233791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da81a73-2c19-42d2-ae43-cdf6c54451b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:58.298191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-6da81a73-2c19-42d2-ae43-cdf6c54451b6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:58.299391 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-6da81a73-2c19-42d2-ae43-cdf6c54451b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:58.311689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:58.394523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:20:58.676324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11d984-fcdf-4411-af60-51d484a2a4a8 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:58.677929 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3e11d984-fcdf-4411-af60-51d484a2a4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:58.781088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11d984-fcdf-4411-af60-51d484a2a4a8 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:58.793563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:58.854719 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf0151-b1a1-4ce8-9e9f-02270cef0ff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:58.855321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf0151-b1a1-4ce8-9e9f-02270cef0ff8 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:58.855603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf0151-b1a1-4ce8-9e9f-02270cef0ff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:58.855877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf0151-b1a1-4ce8-9e9f-02270cef0ff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:58.857517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-afdf0151-b1a1-4ce8-9e9f-02270cef0ff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:58.968039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf0151-b1a1-4ce8-9e9f-02270cef0ff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:58.969513 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf0151-b1a1-4ce8-9e9f-02270cef0ff8 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:58.987691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 721/1446] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:20:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:01.278063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bb687-3140-4fd5-b124-9b7cd1079975 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.278637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bb687-3140-4fd5-b124-9b7cd1079975 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.278637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bb687-3140-4fd5-b124-9b7cd1079975 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.279959 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:01.285208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83c68e-035b-496c-9d7d-41c5af4530d0 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.285660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83c68e-035b-496c-9d7d-41c5af4530d0 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.285798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83c68e-035b-496c-9d7d-41c5af4530d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.333348 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:01.352785 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-5e83c68e-035b-496c-9d7d-41c5af4530d0 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:01.354001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5e83c68e-035b-496c-9d7d-41c5af4530d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:01.449246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:01.736379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430bb687-3140-4fd5-b124-9b7cd1079975 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:01.737981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-430bb687-3140-4fd5-b124-9b7cd1079975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:01.839578 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430bb687-3140-4fd5-b124-9b7cd1079975 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:01.879352 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:01.912606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5db974-df10-4360-8d38-e66dd38cc95b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:01.913211 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5db974-df10-4360-8d38-e66dd38cc95b service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.913489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5db974-df10-4360-8d38-e66dd38cc95b service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.913758 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5db974-df10-4360-8d38-e66dd38cc95b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.916720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1d5db974-df10-4360-8d38-e66dd38cc95b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:02.022494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5db974-df10-4360-8d38-e66dd38cc95b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:02.023978 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5db974-df10-4360-8d38-e66dd38cc95b service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:02.041863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 727/1449] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:04.079344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f773ae8-14df-432a-a11e-8af5a95fb07c None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:04.079909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f773ae8-14df-432a-a11e-8af5a95fb07c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:04.079909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f773ae8-14df-432a-a11e-8af5a95fb07c None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:04.081295 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:04.086527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481e396-1930-45e3-99c8-249f7968cb9d None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:04.086937 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481e396-1930-45e3-99c8-249f7968cb9d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:04.087074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481e396-1930-45e3-99c8-249f7968cb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:04.153553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-6481e396-1930-45e3-99c8-249f7968cb9d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:04.154753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-6481e396-1930-45e3-99c8-249f7968cb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:04.175397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:04.249605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:04.530621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f773ae8-14df-432a-a11e-8af5a95fb07c None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:04.532156 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-4f773ae8-14df-432a-a11e-8af5a95fb07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:04.635233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f773ae8-14df-432a-a11e-8af5a95fb07c None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:04.655357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:04.707369 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596876c6-043b-4db6-ade5-20e4395fada7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:04.707963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596876c6-043b-4db6-ade5-20e4395fada7 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:04.708272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596876c6-043b-4db6-ade5-20e4395fada7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:04.708540 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596876c6-043b-4db6-ade5-20e4395fada7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:04.710120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-596876c6-043b-4db6-ade5-20e4395fada7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:04.816922 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596876c6-043b-4db6-ade5-20e4395fada7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:04.818351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596876c6-043b-4db6-ade5-20e4395fada7 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:04.841515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 724/1452] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:06.908857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372da56c-8791-492d-8126-8c5a4ca7a1bb None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:06.909431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372da56c-8791-492d-8126-8c5a4ca7a1bb None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:06.909431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372da56c-8791-492d-8126-8c5a4ca7a1bb None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:06.910734 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:06.916590 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710b502-4369-4fd2-b1f8-9ed5a0d1c722 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:06.916995 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710b502-4369-4fd2-b1f8-9ed5a0d1c722 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:06.917150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710b502-4369-4fd2-b1f8-9ed5a0d1c722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:06.981418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:06.984062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-a710b502-4369-4fd2-b1f8-9ed5a0d1c722 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:06.985301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a710b502-4369-4fd2-b1f8-9ed5a0d1c722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:07.081188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:07.363847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372da56c-8791-492d-8126-8c5a4ca7a1bb None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:07.365407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-372da56c-8791-492d-8126-8c5a4ca7a1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:07.467263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372da56c-8791-492d-8126-8c5a4ca7a1bb None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:07.515598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:07.540053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfda15-cef8-4ee5-b1f8-cae64f632215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:07.540683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfda15-cef8-4ee5-b1f8-cae64f632215 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:07.540961 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfda15-cef8-4ee5-b1f8-cae64f632215 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:07.541231 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfda15-cef8-4ee5-b1f8-cae64f632215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:07.542802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0cdfda15-cef8-4ee5-b1f8-cae64f632215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:07.650516 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfda15-cef8-4ee5-b1f8-cae64f632215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:07.651970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfda15-cef8-4ee5-b1f8-cae64f632215 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:07.670697 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 730/1455] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:09.636933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555df31-ccd9-46db-841a-f04fe3ebb2b4 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:09.637406 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555df31-ccd9-46db-841a-f04fe3ebb2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:09.637513 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555df31-ccd9-46db-841a-f04fe3ebb2b4 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:09.638828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:09.644363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d0a1b-ec05-4520-88d8-7a12f58a21f0 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:09.644752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d0a1b-ec05-4520-88d8-7a12f58a21f0 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:09.644917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d0a1b-ec05-4520-88d8-7a12f58a21f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:09.713317 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-c85d0a1b-ec05-4520-88d8-7a12f58a21f0 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:09.714548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c85d0a1b-ec05-4520-88d8-7a12f58a21f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:09.746654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:09.814903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:10.132348 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555df31-ccd9-46db-841a-f04fe3ebb2b4 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:10.133892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8555df31-ccd9-46db-841a-f04fe3ebb2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:10.242358 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555df31-ccd9-46db-841a-f04fe3ebb2b4 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:10.270683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:10.315747 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb05eb-4164-477e-8c42-5f656b455483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:10.316378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb05eb-4164-477e-8c42-5f656b455483 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:10.316661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb05eb-4164-477e-8c42-5f656b455483 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:10.316933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb05eb-4164-477e-8c42-5f656b455483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:10.319744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-66fb05eb-4164-477e-8c42-5f656b455483 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:10.430817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb05eb-4164-477e-8c42-5f656b455483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:10.432288 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb05eb-4164-477e-8c42-5f656b455483 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:10.450632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 727/1458] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:21:12.434350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cef1d-7161-4a82-9ad3-9dc968e6c971 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:12.434916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cef1d-7161-4a82-9ad3-9dc968e6c971 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:12.434916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cef1d-7161-4a82-9ad3-9dc968e6c971 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:12.436354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:12.441581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fcf46-f690-4a23-b0f1-437cabfaf722 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:12.441997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fcf46-f690-4a23-b0f1-437cabfaf722 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:12.442143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fcf46-f690-4a23-b0f1-437cabfaf722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:12.510224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3a8fcf46-f690-4a23-b0f1-437cabfaf722 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:12.511355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3a8fcf46-f690-4a23-b0f1-437cabfaf722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:12.519936 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:12.608731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:12.895236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cef1d-7161-4a82-9ad3-9dc968e6c971 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:12.896807 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f49cef1d-7161-4a82-9ad3-9dc968e6c971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:13.002369 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cef1d-7161-4a82-9ad3-9dc968e6c971 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:13.058637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:13.077319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085923f9-9bec-44ed-908f-3ba2451901f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:13.077916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085923f9-9bec-44ed-908f-3ba2451901f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:13.078197 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085923f9-9bec-44ed-908f-3ba2451901f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:13.078468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085923f9-9bec-44ed-908f-3ba2451901f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:13.080052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-085923f9-9bec-44ed-908f-3ba2451901f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:13.187430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085923f9-9bec-44ed-908f-3ba2451901f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:13.188916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085923f9-9bec-44ed-908f-3ba2451901f4 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:13.207050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 733/1461] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:15.150542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93300a8-8771-424d-837b-07d2ec311e4f None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:15.151122 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93300a8-8771-424d-837b-07d2ec311e4f None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:15.151122 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93300a8-8771-424d-837b-07d2ec311e4f None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:15.152620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:15.157838 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0825f6-2e2d-40e5-a241-dd8f10816468 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:15.158305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0825f6-2e2d-40e5-a241-dd8f10816468 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:15.158413 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0825f6-2e2d-40e5-a241-dd8f10816468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:15.224365 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-4f0825f6-2e2d-40e5-a241-dd8f10816468 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:15.225572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4f0825f6-2e2d-40e5-a241-dd8f10816468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:15.263838 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:15.322810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:15.611193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93300a8-8771-424d-837b-07d2ec311e4f None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:15.612762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b93300a8-8771-424d-837b-07d2ec311e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:15.716017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93300a8-8771-424d-837b-07d2ec311e4f None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:15.752787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:15.789550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97578269-53c7-41a1-b41a-849cf2e5fa76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:15.790150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97578269-53c7-41a1-b41a-849cf2e5fa76 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:15.790432 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97578269-53c7-41a1-b41a-849cf2e5fa76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:15.790702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97578269-53c7-41a1-b41a-849cf2e5fa76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:15.792323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-97578269-53c7-41a1-b41a-849cf2e5fa76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:15.904653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97578269-53c7-41a1-b41a-849cf2e5fa76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:15.906103 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97578269-53c7-41a1-b41a-849cf2e5fa76 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:15.924682 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 730/1464] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:17.860254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82b2e5-b796-426a-83a8-a7e450326465 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:17.860765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82b2e5-b796-426a-83a8-a7e450326465 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:17.860861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82b2e5-b796-426a-83a8-a7e450326465 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:17.862502 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:17.870478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666eb178-1f77-41b6-bd6d-7cd3015eb88c None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:17.870478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666eb178-1f77-41b6-bd6d-7cd3015eb88c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:17.870478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666eb178-1f77-41b6-bd6d-7cd3015eb88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:17.934887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-666eb178-1f77-41b6-bd6d-7cd3015eb88c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:17.936085 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-666eb178-1f77-41b6-bd6d-7cd3015eb88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:17.954150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:18.036136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:18.321111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82b2e5-b796-426a-83a8-a7e450326465 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:18.322748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ab82b2e5-b796-426a-83a8-a7e450326465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:18.428039 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82b2e5-b796-426a-83a8-a7e450326465 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:18.494856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:18.501581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a5a32-08ad-4753-801b-abc76a560866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:18.502185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a5a32-08ad-4753-801b-abc76a560866 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:18.502465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a5a32-08ad-4753-801b-abc76a560866 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:18.502739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a5a32-08ad-4753-801b-abc76a560866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:18.504370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-997a5a32-08ad-4753-801b-abc76a560866 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:18.614225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a5a32-08ad-4753-801b-abc76a560866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:18.615187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a5a32-08ad-4753-801b-abc76a560866 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:18.634135 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 736/1467] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:20.549075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea684c-9ef4-40a3-8b23-f3285aaf1b5b None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.549640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea684c-9ef4-40a3-8b23-f3285aaf1b5b None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.549640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea684c-9ef4-40a3-8b23-f3285aaf1b5b None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.550977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:20.556206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d012596-ede0-4211-bfd8-4bc361e89651 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.556623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d012596-ede0-4211-bfd8-4bc361e89651 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.556764 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d012596-ede0-4211-bfd8-4bc361e89651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.623894 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-8d012596-ede0-4211-bfd8-4bc361e89651 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:20.625151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8d012596-ede0-4211-bfd8-4bc361e89651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:20.673131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:20.722479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:21.004860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea684c-9ef4-40a3-8b23-f3285aaf1b5b None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:21.006433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7dea684c-9ef4-40a3-8b23-f3285aaf1b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:21.107636 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea684c-9ef4-40a3-8b23-f3285aaf1b5b None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:21.156110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:21.181716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69e3ec-19b7-4ff6-aeb7-8ddb9f30922e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:21.182321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69e3ec-19b7-4ff6-aeb7-8ddb9f30922e service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:21.182601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69e3ec-19b7-4ff6-aeb7-8ddb9f30922e service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:21.182876 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69e3ec-19b7-4ff6-aeb7-8ddb9f30922e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:21.184500 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3c69e3ec-19b7-4ff6-aeb7-8ddb9f30922e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:21.291514 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69e3ec-19b7-4ff6-aeb7-8ddb9f30922e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:21.292900 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69e3ec-19b7-4ff6-aeb7-8ddb9f30922e service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:21.310980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 733/1470] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:30.720623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=22 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:31.308710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=66 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:40.780958 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d9718-ec3b-4117-9d14-3b42a43181f9 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:40.781519 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d9718-ec3b-4117-9d14-3b42a43181f9 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:40.781519 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d9718-ec3b-4117-9d14-3b42a43181f9 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:40.782864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 738/1471] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:40.788481 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb7832-b9f9-40fc-b4e2-348029752b6a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:40.788995 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb7832-b9f9-40fc-b4e2-348029752b6a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:40.789105 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb7832-b9f9-40fc-b4e2-348029752b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:40.856251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-87cb7832-b9f9-40fc-b4e2-348029752b6a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:40.857457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-87cb7832-b9f9-40fc-b4e2-348029752b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:40.954807 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 734/1472] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:41.253617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadb4d8-f9a9-4623-bbdd-a50426352b5a None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:41.253906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadb4d8-f9a9-4623-bbdd-a50426352b5a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:41.254182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadb4d8-f9a9-4623-bbdd-a50426352b5a None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:41.255504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 739/1473] 10.222.0.85 () {54 vars in 967 bytes} [Mon Aug 29 20:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:41.260997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a3919-d6c7-4d2d-968d-20a90a8b9f12 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:41.261284 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a3919-d6c7-4d2d-968d-20a90a8b9f12 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:41.261584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a3919-d6c7-4d2d-968d-20a90a8b9f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:41.339256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-5f1a3919-d6c7-4d2d-968d-20a90a8b9f12 None None] MFA Rules not processed for user `a7e768fdd1054e40b6b4910e39a6e533`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:41.340505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5f1a3919-d6c7-4d2d-968d-20a90a8b9f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:41.361155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:41.416781 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 735/1474] 10.222.0.85 () {58 vars in 1064 bytes} [Mon Aug 29 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:41.424310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadb4d8-f9a9-4623-bbdd-a50426352b5a None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:41.426003 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-daadb4d8-f9a9-4623-bbdd-a50426352b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:41.530296 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadb4d8-f9a9-4623-bbdd-a50426352b5a None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:41.603109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b448aba-1d13-46a7-a175-705f2ee02285 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7e768fdd1054e40b6b4910e39a6e533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:41.603713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b448aba-1d13-46a7-a175-705f2ee02285 service glance] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:41.603996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b448aba-1d13-46a7-a175-705f2ee02285 service glance] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:41.604307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b448aba-1d13-46a7-a175-705f2ee02285 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:41.605908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2b448aba-1d13-46a7-a175-705f2ee02285 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:41.662071 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:41.716662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b448aba-1d13-46a7-a175-705f2ee02285 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:41.719099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b448aba-1d13-46a7-a175-705f2ee02285 service glance] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:41.737681 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 740/1475] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:43.728860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa64bf-e976-4c6f-82fd-d4c65daddce2 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:43.729420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa64bf-e976-4c6f-82fd-d4c65daddce2 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:43.729420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa64bf-e976-4c6f-82fd-d4c65daddce2 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:43.730805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 736/1476] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:43.736084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7374c-26d0-473c-8784-2f37c29b38d4 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:43.736545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7374c-26d0-473c-8784-2f37c29b38d4 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:43.736671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7374c-26d0-473c-8784-2f37c29b38d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:43.801736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f8c7374c-26d0-473c-8784-2f37c29b38d4 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:43.802920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f8c7374c-26d0-473c-8784-2f37c29b38d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:43.899679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 741/1477] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:44.172082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa64bf-e976-4c6f-82fd-d4c65daddce2 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:44.173673 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5cfa64bf-e976-4c6f-82fd-d4c65daddce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:44.298644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa64bf-e976-4c6f-82fd-d4c65daddce2 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:44.346460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:44.391038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c22e4-7da0-420d-bcb3-3cdefde50ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:44.391641 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c22e4-7da0-420d-bcb3-3cdefde50ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:44.391921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c22e4-7da0-420d-bcb3-3cdefde50ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:44.392210 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c22e4-7da0-420d-bcb3-3cdefde50ef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:44.392849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c22e4-7da0-420d-bcb3-3cdefde50ef6 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:44.394893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: WARNING py.warnings [None req-043c22e4-7da0-420d-bcb3-3cdefde50ef6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:21:44.394893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: warnings.warn(msg) Aug 29 20:21:44.394893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: Aug 29 20:21:44.396038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c22e4-7da0-420d-bcb3-3cdefde50ef6 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:44.415044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 737/1478] 10.222.0.85 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:44 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:46.342032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb29843-b3b0-46c0-92f5-9a729a7e4afc None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:46.342639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb29843-b3b0-46c0-92f5-9a729a7e4afc None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:46.342639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb29843-b3b0-46c0-92f5-9a729a7e4afc None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:46.343968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 742/1479] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:46.349301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfff02-666d-4442-9fdb-761543481f23 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:46.349714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfff02-666d-4442-9fdb-761543481f23 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:46.349860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfff02-666d-4442-9fdb-761543481f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:46.415658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-56bfff02-666d-4442-9fdb-761543481f23 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:46.416888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-56bfff02-666d-4442-9fdb-761543481f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:46.632798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 738/1480] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:46.929003 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb29843-b3b0-46c0-92f5-9a729a7e4afc None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:46.930503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-adb29843-b3b0-46c0-92f5-9a729a7e4afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:46.940292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:47.075060 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb29843-b3b0-46c0-92f5-9a729a7e4afc None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:47.274467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:47.306221 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ee965-e91c-4d4f-9293-5438c4244343 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:47.306827 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ee965-e91c-4d4f-9293-5438c4244343 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:47.307101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ee965-e91c-4d4f-9293-5438c4244343 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:47.307372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ee965-e91c-4d4f-9293-5438c4244343 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:47.309005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-454ee965-e91c-4d4f-9293-5438c4244343 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:47.398332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ee965-e91c-4d4f-9293-5438c4244343 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:47.399918 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ee965-e91c-4d4f-9293-5438c4244343 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:47.417513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 743/1481] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:49.509484 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e03b4-8972-46cf-970e-7cf27c3936c9 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:49.510076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e03b4-8972-46cf-970e-7cf27c3936c9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:49.510076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e03b4-8972-46cf-970e-7cf27c3936c9 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:49.511366 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 739/1482] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:49.516629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a060f-0381-4089-8311-d46de966aa5c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:49.517044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a060f-0381-4089-8311-d46de966aa5c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:49.517177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a060f-0381-4089-8311-d46de966aa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:49.585375 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-287a060f-0381-4089-8311-d46de966aa5c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:49.586591 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-287a060f-0381-4089-8311-d46de966aa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:49.633444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:49.683870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 744/1483] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:49.968665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e03b4-8972-46cf-970e-7cf27c3936c9 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:49.970199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3f5e03b4-8972-46cf-970e-7cf27c3936c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:49.993587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:50.074616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e03b4-8972-46cf-970e-7cf27c3936c9 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:50.146344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6587c1c-51da-4dd3-a7b0-a43b9303bc6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:50.146935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6587c1c-51da-4dd3-a7b0-a43b9303bc6a service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:50.147212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6587c1c-51da-4dd3-a7b0-a43b9303bc6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:50.147481 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6587c1c-51da-4dd3-a7b0-a43b9303bc6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:50.149202 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d6587c1c-51da-4dd3-a7b0-a43b9303bc6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:50.221490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:50.256946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6587c1c-51da-4dd3-a7b0-a43b9303bc6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:50.258600 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6587c1c-51da-4dd3-a7b0-a43b9303bc6a service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:50.277068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 740/1484] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:52.219931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31840a1-5624-48e8-a74e-8fc1cc6cc12c None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:52.220545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31840a1-5624-48e8-a74e-8fc1cc6cc12c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:52.220545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31840a1-5624-48e8-a74e-8fc1cc6cc12c None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:52.221879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 745/1485] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:52.228911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d38e3-ca69-4ec8-947f-6c9ccfd7adb9 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:52.229344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d38e3-ca69-4ec8-947f-6c9ccfd7adb9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:52.229469 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d38e3-ca69-4ec8-947f-6c9ccfd7adb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:52.294781 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-c81d38e3-ca69-4ec8-947f-6c9ccfd7adb9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:52.295972 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c81d38e3-ca69-4ec8-947f-6c9ccfd7adb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:52.392737 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 741/1486] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:52.687418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31840a1-5624-48e8-a74e-8fc1cc6cc12c None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:52.689024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b31840a1-5624-48e8-a74e-8fc1cc6cc12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:52.792043 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31840a1-5624-48e8-a74e-8fc1cc6cc12c None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:52.864532 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:52.866732 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be12f42-f0c8-4cee-982c-7ca26116c077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:52.867337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be12f42-f0c8-4cee-982c-7ca26116c077 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:52.867617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be12f42-f0c8-4cee-982c-7ca26116c077 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:52.867888 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be12f42-f0c8-4cee-982c-7ca26116c077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:52.869514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0be12f42-f0c8-4cee-982c-7ca26116c077 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:52.977358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be12f42-f0c8-4cee-982c-7ca26116c077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:52.978784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be12f42-f0c8-4cee-982c-7ca26116c077 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:52.996651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 746/1487] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:55.019381 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8a81d-2846-41a4-9df8-948fb30a804d None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.019969 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8a81d-2846-41a4-9df8-948fb30a804d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.019969 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8a81d-2846-41a4-9df8-948fb30a804d None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.021893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 742/1488] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:21:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:55.027036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc6d0-6492-4234-b4b8-c1063346fd1c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.027469 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc6d0-6492-4234-b4b8-c1063346fd1c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.027577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc6d0-6492-4234-b4b8-c1063346fd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.092426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-498cc6d0-6492-4234-b4b8-c1063346fd1c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:55.093626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-498cc6d0-6492-4234-b4b8-c1063346fd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:55.145373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:55.188655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 747/1489] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:55.476093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8a81d-2846-41a4-9df8-948fb30a804d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:55.477652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b8a8a81d-2846-41a4-9df8-948fb30a804d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:55.513815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:55.580332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8a81d-2846-41a4-9df8-948fb30a804d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:55.653806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6f929-9a74-4d17-80bb-e7eb9cb8abe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:55.654412 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6f929-9a74-4d17-80bb-e7eb9cb8abe1 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.654691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6f929-9a74-4d17-80bb-e7eb9cb8abe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.654963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6f929-9a74-4d17-80bb-e7eb9cb8abe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.656641 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-09c6f929-9a74-4d17-80bb-e7eb9cb8abe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:55.736459 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:55.764523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6f929-9a74-4d17-80bb-e7eb9cb8abe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:55.765958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6f929-9a74-4d17-80bb-e7eb9cb8abe1 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:55.783756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 743/1490] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:21:58.265861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2c6fc-f329-4456-9899-fdfe8284059d None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:58.266445 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2c6fc-f329-4456-9899-fdfe8284059d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:58.266445 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2c6fc-f329-4456-9899-fdfe8284059d None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:58.267755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 748/1491] 10.222.0.85 () {54 vars in 927 bytes} [Mon Aug 29 20:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:58.273149 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ba3da-f37f-4ec4-b54b-73e85b348d11 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:58.273557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ba3da-f37f-4ec4-b54b-73e85b348d11 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:58.273699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ba3da-f37f-4ec4-b54b-73e85b348d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:58.338347 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-5b4ba3da-f37f-4ec4-b54b-73e85b348d11 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:58.339540 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5b4ba3da-f37f-4ec4-b54b-73e85b348d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:58.436625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 744/1492] 10.222.0.85 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:21:58.730232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2c6fc-f329-4456-9899-fdfe8284059d None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:58.731796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b2c2c6fc-f329-4456-9899-fdfe8284059d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:58.832765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2c6fc-f329-4456-9899-fdfe8284059d None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:58.904421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b8592-2748-4893-acce-a15a32d7d937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:58.905017 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b8592-2748-4893-acce-a15a32d7d937 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:58.905313 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b8592-2748-4893-acce-a15a32d7d937 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:58.905579 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b8592-2748-4893-acce-a15a32d7d937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:58.907142 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f64b8592-2748-4893-acce-a15a32d7d937 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:58.916533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:59.012746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b8592-2748-4893-acce-a15a32d7d937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:59.014184 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b8592-2748-4893-acce-a15a32d7d937 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:59.032680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 749/1493] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:22:08.434808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=82 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:09.030445 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=50 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:22.320051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d62736-d99c-4fa5-bd5b-ef5880753c00 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:22.320700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d62736-d99c-4fa5-bd5b-ef5880753c00 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:22.320700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d62736-d99c-4fa5-bd5b-ef5880753c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:22.399806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-08d62736-d99c-4fa5-bd5b-ef5880753c00 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:22.401060 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-08d62736-d99c-4fa5-bd5b-ef5880753c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:22.496295 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 745/1494] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:23:22.504973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b78db-7c3e-4bba-b5ef-afdd77719caa None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:22.505469 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b78db-7c3e-4bba-b5ef-afdd77719caa None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:22.505578 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b78db-7c3e-4bba-b5ef-afdd77719caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:22.579323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-120b78db-7c3e-4bba-b5ef-afdd77719caa None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:22.581067 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-120b78db-7c3e-4bba-b5ef-afdd77719caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:22.674509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 750/1495] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:23:22.679200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d62736-d99c-4fa5-bd5b-ef5880753c00 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:22.680440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-08d62736-d99c-4fa5-bd5b-ef5880753c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:22.692997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d62736-d99c-4fa5-bd5b-ef5880753c00 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:22.753517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:22.786965 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5a521-90a2-4779-80aa-75225f92d0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:22.787800 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5a521-90a2-4779-80aa-75225f92d0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:22.788305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5a521-90a2-4779-80aa-75225f92d0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:22.788857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5a521-90a2-4779-80aa-75225f92d0c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:22.789510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5a521-90a2-4779-80aa-75225f92d0c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:22.789814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5a521-90a2-4779-80aa-75225f92d0c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:22.790826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: WARNING py.warnings [None req-b4c5a521-90a2-4779-80aa-75225f92d0c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:23:22.790826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: warnings.warn(msg) Aug 29 20:23:22.790826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: Aug 29 20:23:22.791704 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5a521-90a2-4779-80aa-75225f92d0c5 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:22.802718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 746/1496] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:22.810981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b78db-7c3e-4bba-b5ef-afdd77719caa None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:22.820557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b78db-7c3e-4bba-b5ef-afdd77719caa None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:22.880633 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:22.912963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535abadd-9a30-4788-a3ca-eeef4a0fec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:22.913559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535abadd-9a30-4788-a3ca-eeef4a0fec50 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:22.913832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535abadd-9a30-4788-a3ca-eeef4a0fec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:22.914098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535abadd-9a30-4788-a3ca-eeef4a0fec50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:22.914724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535abadd-9a30-4788-a3ca-eeef4a0fec50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:22.916482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: WARNING py.warnings [None req-535abadd-9a30-4788-a3ca-eeef4a0fec50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:23:22.916482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: warnings.warn(msg) Aug 29 20:23:22.916482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: Aug 29 20:23:22.917692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535abadd-9a30-4788-a3ca-eeef4a0fec50 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:23.046489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 751/1497] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:23:22 2022] POST /identity/v3/projects => generated 379 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:23:23.050678 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5a521-90a2-4779-80aa-75225f92d0c5 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:23.052267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b4c5a521-90a2-4779-80aa-75225f92d0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:23.084783 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5a521-90a2-4779-80aa-75225f92d0c5 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:23.175920 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6ae04-d721-49bc-817c-2651fadf6700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:23.176534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6ae04-d721-49bc-817c-2651fadf6700 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:23.176813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6ae04-d721-49bc-817c-2651fadf6700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:23.177086 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6ae04-d721-49bc-817c-2651fadf6700 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:23.177716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6ae04-d721-49bc-817c-2651fadf6700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:23.178725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: WARNING py.warnings [None req-1dc6ae04-d721-49bc-817c-2651fadf6700 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:23:23.178725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: warnings.warn(msg) Aug 29 20:23:23.178725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: Aug 29 20:23:23.179926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6ae04-d721-49bc-817c-2651fadf6700 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:23.349207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 747/1498] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:23:23 2022] POST /identity/v3/users => generated 290 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:23:23.353521 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535abadd-9a30-4788-a3ca-eeef4a0fec50 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:23.355136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-535abadd-9a30-4788-a3ca-eeef4a0fec50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:23.418493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535abadd-9a30-4788-a3ca-eeef4a0fec50 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:23.504536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:23.515652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90abc56-c771-4f87-8860-b87f04d96c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:23.520907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90abc56-c771-4f87-8860-b87f04d96c1f admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:23.520907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90abc56-c771-4f87-8860-b87f04d96c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:23.520907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90abc56-c771-4f87-8860-b87f04d96c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:23.520907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90abc56-c771-4f87-8860-b87f04d96c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:23.520907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: WARNING py.warnings [None req-b90abc56-c771-4f87-8860-b87f04d96c1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:23:23.520907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: warnings.warn(msg) Aug 29 20:23:23.520907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: Aug 29 20:23:23.521456 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90abc56-c771-4f87-8860-b87f04d96c1f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:23.533440 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 752/1499] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:23:23.537511 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6ae04-d721-49bc-817c-2651fadf6700 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:23.539096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-1dc6ae04-d721-49bc-817c-2651fadf6700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:23.551118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6ae04-d721-49bc-817c-2651fadf6700 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:23.577004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:23.649234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d077fd1-0562-4e33-abee-cadbf068f14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:23.649842 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d077fd1-0562-4e33-abee-cadbf068f14c admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:23.650121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d077fd1-0562-4e33-abee-cadbf068f14c admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:23.650390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d077fd1-0562-4e33-abee-cadbf068f14c admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:23.651034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d077fd1-0562-4e33-abee-cadbf068f14c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:23.652047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: WARNING py.warnings [None req-3d077fd1-0562-4e33-abee-cadbf068f14c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:23:23.652047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: warnings.warn(msg) Aug 29 20:23:23.652047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: Aug 29 20:23:23.652981 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d077fd1-0562-4e33-abee-cadbf068f14c admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:23.665617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 748/1500] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:23:23.669924 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90abc56-c771-4f87-8860-b87f04d96c1f admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:23.671453 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b90abc56-c771-4f87-8860-b87f04d96c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:23.705105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90abc56-c771-4f87-8860-b87f04d96c1f admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:23.798725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad6732-43cc-4e19-926f-58d064d1774d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:23.799339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad6732-43cc-4e19-926f-58d064d1774d admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:23.799575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad6732-43cc-4e19-926f-58d064d1774d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:23.799842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad6732-43cc-4e19-926f-58d064d1774d admin admin] PATH_INFO: `/v3/projects/45dce30c5a104a0aa1ccc71302086eff/users/becb558ebf6a4a7da3f5361e321e3993/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:23.821747 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad6732-43cc-4e19-926f-58d064d1774d admin admin] RBAC: Authorizing `identity:create_grant(project_id=45dce30c5a104a0aa1ccc71302086eff, user_id=becb558ebf6a4a7da3f5361e321e3993, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:23.822738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: WARNING py.warnings [None req-56ad6732-43cc-4e19-926f-58d064d1774d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:23:23.822738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: warnings.warn(msg) Aug 29 20:23:23.822738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: Aug 29 20:23:23.824721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad6732-43cc-4e19-926f-58d064d1774d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:24.144637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 753/1501] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:23:23 2022] PUT /identity/v3/projects/45dce30c5a104a0aa1ccc71302086eff/users/becb558ebf6a4a7da3f5361e321e3993/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:23:24.156759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7ebbb-a49e-4bf1-bbf3-86e08f452803 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:24.157046 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7ebbb-a49e-4bf1-bbf3-86e08f452803 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:24.157320 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7ebbb-a49e-4bf1-bbf3-86e08f452803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:24.216341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-14f7ebbb-a49e-4bf1-bbf3-86e08f452803 None None] MFA Rules not processed for user `becb558ebf6a4a7da3f5361e321e3993`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:24.217551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-14f7ebbb-a49e-4bf1-bbf3-86e08f452803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:24.293365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 749/1502] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:23:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:23:24.306289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9f052-c807-4435-88e0-f14dd8387411 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:24.306582 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9f052-c807-4435-88e0-f14dd8387411 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:24.306863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9f052-c807-4435-88e0-f14dd8387411 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:24.308220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 754/1503] 10.222.0.85 () {54 vars in 964 bytes} [Mon Aug 29 20:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:23:24.313288 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc73e-a61d-4ac7-8f17-3ead0382cc5d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:24.313603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc73e-a61d-4ac7-8f17-3ead0382cc5d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:24.314423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc73e-a61d-4ac7-8f17-3ead0382cc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:24.389338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-7d5bc73e-a61d-4ac7-8f17-3ead0382cc5d None None] MFA Rules not processed for user `9d90a7ed44fd4a1fa94b9d44939df425`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:24.390548 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7d5bc73e-a61d-4ac7-8f17-3ead0382cc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:24.466596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 750/1504] 10.222.0.85 () {58 vars in 1061 bytes} [Mon Aug 29 20:23:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:23:24.474356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9f052-c807-4435-88e0-f14dd8387411 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:24.507103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-fba9f052-c807-4435-88e0-f14dd8387411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:24.526510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:24.551327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:24.579254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9f052-c807-4435-88e0-f14dd8387411 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:24.651019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48e782-1799-437d-be98-813b2909d006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:24.651620 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48e782-1799-437d-be98-813b2909d006 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:24.651898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48e782-1799-437d-be98-813b2909d006 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:24.652189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48e782-1799-437d-be98-813b2909d006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:24.653774 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-6f48e782-1799-437d-be98-813b2909d006 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:24.750341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48e782-1799-437d-be98-813b2909d006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:24.752267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48e782-1799-437d-be98-813b2909d006 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:24.771995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 755/1505] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:23:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:23:24.815809 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a30efc-fd59-4507-a4a3-75df24feb515 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:24.816096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a30efc-fd59-4507-a4a3-75df24feb515 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:24.816401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a30efc-fd59-4507-a4a3-75df24feb515 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:24.817661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 751/1506] 10.222.0.85 () {52 vars in 830 bytes} [Mon Aug 29 20:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:23:24.821561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48e782-1799-437d-be98-813b2909d006 service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:24.822710 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-6f48e782-1799-437d-be98-813b2909d006 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:24.906862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:24.919792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48e782-1799-437d-be98-813b2909d006 service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:25.012014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:25.012590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:25.012867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:25.013134 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] PATH_INFO: `/v3/users/becb558ebf6a4a7da3f5361e321e3993` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:25.025259 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=becb558ebf6a4a7da3f5361e321e3993)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:25.026271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: WARNING py.warnings [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:23:25.026271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: warnings.warn(msg) Aug 29 20:23:25.026271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: Aug 29 20:23:25.027497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:25.085947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:25.380701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'becb558ebf6a4a7da3f5361e321e3993'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:23:25.381163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'becb558ebf6a4a7da3f5361e321e3993'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:23:25.395243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'becb558ebf6a4a7da3f5361e321e3993'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:23:25.497148 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'becb558ebf6a4a7da3f5361e321e3993'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:23:25.497581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'becb558ebf6a4a7da3f5361e321e3993'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:23:25.527215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 756/1507] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:23:24 2022] DELETE /identity/v3/users/becb558ebf6a4a7da3f5361e321e3993 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:23:25.531778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c9117-26a8-4098-8e0b-8a813de2c206 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:25.532291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c9117-26a8-4098-8e0b-8a813de2c206 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:25.532389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c9117-26a8-4098-8e0b-8a813de2c206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:25.588911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3b4c9117-26a8-4098-8e0b-8a813de2c206 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:25.590218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3b4c9117-26a8-4098-8e0b-8a813de2c206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:25.667383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:25.680438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 752/1508] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:23:25.690181 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:25.692719 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:25.793711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0b011-436b-4a71-a958-e577e262b4d1 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:25.847722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:25.866930 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4ed7d-1217-498a-be35-19c24723338c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:25.867559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4ed7d-1217-498a-be35-19c24723338c service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:25.867834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4ed7d-1217-498a-be35-19c24723338c service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:25.868103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4ed7d-1217-498a-be35-19c24723338c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:25.869917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-37b4ed7d-1217-498a-be35-19c24723338c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:25.978338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4ed7d-1217-498a-be35-19c24723338c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:25.979699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4ed7d-1217-498a-be35-19c24723338c service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:25.999294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 757/1509] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:23:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:23:26.636230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c9117-26a8-4098-8e0b-8a813de2c206 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:26.637656 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3b4c9117-26a8-4098-8e0b-8a813de2c206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:26.650011 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c9117-26a8-4098-8e0b-8a813de2c206 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:26.746183 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57561518-4777-4e43-9a02-1451eaca8a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:26.746811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57561518-4777-4e43-9a02-1451eaca8a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:26.747095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57561518-4777-4e43-9a02-1451eaca8a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:26.747368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57561518-4777-4e43-9a02-1451eaca8a0d admin admin] PATH_INFO: `/v3/projects/45dce30c5a104a0aa1ccc71302086eff` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:26.757196 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57561518-4777-4e43-9a02-1451eaca8a0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=45dce30c5a104a0aa1ccc71302086eff)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:26.758267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: WARNING py.warnings [None req-57561518-4777-4e43-9a02-1451eaca8a0d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:23:26.758267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: warnings.warn(msg) Aug 29 20:23:26.758267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: Aug 29 20:23:26.759461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57561518-4777-4e43-9a02-1451eaca8a0d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:26.889154 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-57561518-4777-4e43-9a02-1451eaca8a0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45dce30c5a104a0aa1ccc71302086eff', 'request_id': 'req-57561518-4777-4e43-9a02-1451eaca8a0d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:23:26.903027 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-57561518-4777-4e43-9a02-1451eaca8a0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45dce30c5a104a0aa1ccc71302086eff', 'request_id': 'req-57561518-4777-4e43-9a02-1451eaca8a0d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:23:26.909562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-57561518-4777-4e43-9a02-1451eaca8a0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45dce30c5a104a0aa1ccc71302086eff', 'request_id': 'req-57561518-4777-4e43-9a02-1451eaca8a0d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:23:27.045696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-57561518-4777-4e43-9a02-1451eaca8a0d admin admin] The token cache is being invalidate because project 45dce30c5a104a0aa1ccc71302086eff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:23:27.045977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-57561518-4777-4e43-9a02-1451eaca8a0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:23:27.046398 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-57561518-4777-4e43-9a02-1451eaca8a0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:23:27.047804 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 753/1510] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:23:26 2022] DELETE /identity/v3/projects/45dce30c5a104a0aa1ccc71302086eff => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:23:35.996838 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=62 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:37.043660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:00.063263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c39d9-995f-4883-b080-faf9e0854812 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:00.063878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c39d9-995f-4883-b080-faf9e0854812 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:00.063878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c39d9-995f-4883-b080-faf9e0854812 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:00.065206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:00.070020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc06d-1ed2-411e-8cbe-d99c3866991c None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:00.070472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc06d-1ed2-411e-8cbe-d99c3866991c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:00.070591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc06d-1ed2-411e-8cbe-d99c3866991c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:00.149895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-5c6dc06d-1ed2-411e-8cbe-d99c3866991c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:00.151071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5c6dc06d-1ed2-411e-8cbe-d99c3866991c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:00.255335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:00.408651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c39d9-995f-4883-b080-faf9e0854812 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:00.410867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7d5c39d9-995f-4883-b080-faf9e0854812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:00.527396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c39d9-995f-4883-b080-faf9e0854812 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:00.532833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09697f-6a40-463e-b441-eb7b552cf50a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:00.533425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09697f-6a40-463e-b441-eb7b552cf50a service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:00.533702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09697f-6a40-463e-b441-eb7b552cf50a service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:00.533972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09697f-6a40-463e-b441-eb7b552cf50a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:00.541740 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09697f-6a40-463e-b441-eb7b552cf50a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:00.543335 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09697f-6a40-463e-b441-eb7b552cf50a service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:00.553099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 759/1513] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:01.382175 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4579c-ec16-4c5e-a600-83e27aea73b4 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:01.383042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4579c-ec16-4c5e-a600-83e27aea73b4 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:01.383042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4579c-ec16-4c5e-a600-83e27aea73b4 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:01.384642 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:01.389955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c7d4f-d58f-4859-80a3-b27fceff1b28 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:01.390410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c7d4f-d58f-4859-80a3-b27fceff1b28 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:01.390520 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c7d4f-d58f-4859-80a3-b27fceff1b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:01.410763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f58c7d4f-d58f-4859-80a3-b27fceff1b28 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:01.411974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f58c7d4f-d58f-4859-80a3-b27fceff1b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:01.419920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:25:01.571657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4579c-ec16-4c5e-a600-83e27aea73b4 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:01.586515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4579c-ec16-4c5e-a600-83e27aea73b4 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:01.591891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f04fa-ce56-4306-981d-3b0aad6cf63e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:01.592519 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f04fa-ce56-4306-981d-3b0aad6cf63e service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:01.592819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f04fa-ce56-4306-981d-3b0aad6cf63e service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:01.593070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f04fa-ce56-4306-981d-3b0aad6cf63e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:01.601035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f04fa-ce56-4306-981d-3b0aad6cf63e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:01.602622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f04fa-ce56-4306-981d-3b0aad6cf63e service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:01.612068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 756/1516] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:02.317032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d3f1e-6106-4e40-83c3-b3bfbdde4e2c None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:02.317298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d3f1e-6106-4e40-83c3-b3bfbdde4e2c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:02.317569 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d3f1e-6106-4e40-83c3-b3bfbdde4e2c None None] PATH_INFO: `/v3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:02.318935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 761/1517] 10.222.0.84 () {54 vars in 925 bytes} [Mon Aug 29 20:25:02 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:25:02.326284 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bd977-fe25-49f1-83b7-b7ec83fdd698 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:02.327223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bd977-fe25-49f1-83b7-b7ec83fdd698 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:02.327510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bd977-fe25-49f1-83b7-b7ec83fdd698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:02.387614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-db1bd977-fe25-49f1-83b7-b7ec83fdd698 None None] MFA Rules not processed for user `5288af2dbc15438c8b209b0ba2454cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:02.388848 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-db1bd977-fe25-49f1-83b7-b7ec83fdd698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:02.406355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:02.460211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 757/1518] 10.222.0.84 () {58 vars in 1015 bytes} [Mon Aug 29 20:25:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:11.405933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=56 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:11.812257 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e689b28-d6af-4d52-a3df-2ae88bd0ae73 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:11.812582 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e689b28-d6af-4d52-a3df-2ae88bd0ae73 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:11.812908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e689b28-d6af-4d52-a3df-2ae88bd0ae73 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:11.814527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 762/1519] 10.222.0.85 () {54 vars in 961 bytes} [Mon Aug 29 20:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:11.819840 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b595ab-5bf6-4312-99d8-af39ef6ff315 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:11.820338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b595ab-5bf6-4312-99d8-af39ef6ff315 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:11.820449 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b595ab-5bf6-4312-99d8-af39ef6ff315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:11.840272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-22b595ab-5bf6-4312-99d8-af39ef6ff315 None None] MFA Rules not processed for user `5288af2dbc15438c8b209b0ba2454cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:11.841470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-22b595ab-5bf6-4312-99d8-af39ef6ff315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:11.850644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 758/1520] 10.222.0.85 () {58 vars in 1058 bytes} [Mon Aug 29 20:25:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:25:11.858296 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e689b28-d6af-4d52-a3df-2ae88bd0ae73 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:11.871698 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e689b28-d6af-4d52-a3df-2ae88bd0ae73 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:11.876595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e096d-5c8a-4432-9d15-dd242eb3c1c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:11.877189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e096d-5c8a-4432-9d15-dd242eb3c1c1 service placement] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:11.877483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e096d-5c8a-4432-9d15-dd242eb3c1c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:11.877759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e096d-5c8a-4432-9d15-dd242eb3c1c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:11.886070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e096d-5c8a-4432-9d15-dd242eb3c1c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:11.887661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e096d-5c8a-4432-9d15-dd242eb3c1c1 service placement] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:11.898084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 763/1521] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:12.830219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7fa9d-8c8e-4c30-aedc-b9de8a29bb8d None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:12.830787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7fa9d-8c8e-4c30-aedc-b9de8a29bb8d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:12.830787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7fa9d-8c8e-4c30-aedc-b9de8a29bb8d None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:12.832114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:12.837015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb04f0-be7d-4383-bc2a-c6a45b1d9529 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:12.837517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb04f0-be7d-4383-bc2a-c6a45b1d9529 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:12.837617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb04f0-be7d-4383-bc2a-c6a45b1d9529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:12.857572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-0cfb04f0-be7d-4383-bc2a-c6a45b1d9529 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:12.861826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0cfb04f0-be7d-4383-bc2a-c6a45b1d9529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:12.867394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:13.028367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7fa9d-8c8e-4c30-aedc-b9de8a29bb8d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:13.043419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7fa9d-8c8e-4c30-aedc-b9de8a29bb8d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:13.049599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df15be-6711-4ac4-b261-7bdf4c57f263 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:13.050190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df15be-6711-4ac4-b261-7bdf4c57f263 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:13.050474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df15be-6711-4ac4-b261-7bdf4c57f263 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:13.050749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df15be-6711-4ac4-b261-7bdf4c57f263 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:13.059162 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df15be-6711-4ac4-b261-7bdf4c57f263 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:13.060793 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df15be-6711-4ac4-b261-7bdf4c57f263 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:13.070318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 760/1524] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:13.903833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bdeb0-e6d4-4b5b-a86e-2ebb9360648b None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:13.905031 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bdeb0-e6d4-4b5b-a86e-2ebb9360648b None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:13.905031 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bdeb0-e6d4-4b5b-a86e-2ebb9360648b None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:13.906349 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:13.911285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bf7e1-0b65-4d55-83e5-abd6925b0cf1 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:13.911720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bf7e1-0b65-4d55-83e5-abd6925b0cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:13.911818 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bf7e1-0b65-4d55-83e5-abd6925b0cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:13.931166 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-485bf7e1-0b65-4d55-83e5-abd6925b0cf1 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:13.932468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-485bf7e1-0b65-4d55-83e5-abd6925b0cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:13.940277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:14.093388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0bdeb0-e6d4-4b5b-a86e-2ebb9360648b None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:14.107387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0bdeb0-e6d4-4b5b-a86e-2ebb9360648b None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:14.112751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36a207-5d65-4d41-af09-b46e100cc906 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:14.113342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36a207-5d65-4d41-af09-b46e100cc906 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:14.113624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36a207-5d65-4d41-af09-b46e100cc906 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:14.113898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36a207-5d65-4d41-af09-b46e100cc906 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:14.122210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36a207-5d65-4d41-af09-b46e100cc906 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:14.123858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36a207-5d65-4d41-af09-b46e100cc906 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:14.133110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 766/1527] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:18.376603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d7a46-7999-462f-a6d7-d1f653419cbb None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:18.377175 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d7a46-7999-462f-a6d7-d1f653419cbb None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:18.377175 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d7a46-7999-462f-a6d7-d1f653419cbb None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:18.378519 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:18.383681 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0caf1-1f17-4ec5-b4a3-baf8201a0275 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:18.384084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0caf1-1f17-4ec5-b4a3-baf8201a0275 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:18.384268 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0caf1-1f17-4ec5-b4a3-baf8201a0275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:18.404641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-e2c0caf1-1f17-4ec5-b4a3-baf8201a0275 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:18.405861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e2c0caf1-1f17-4ec5-b4a3-baf8201a0275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:18.413929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:18.701880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d7a46-7999-462f-a6d7-d1f653419cbb None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:18.703580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bd9d7a46-7999-462f-a6d7-d1f653419cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:18.780608 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d7a46-7999-462f-a6d7-d1f653419cbb None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:18.784154 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522e5da-8f1a-4921-8744-b6e5ba437c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:18.784773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522e5da-8f1a-4921-8744-b6e5ba437c7c service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:18.785055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522e5da-8f1a-4921-8744-b6e5ba437c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:18.785329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522e5da-8f1a-4921-8744-b6e5ba437c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:18.794079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522e5da-8f1a-4921-8744-b6e5ba437c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:18.795508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522e5da-8f1a-4921-8744-b6e5ba437c7c service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:18.805417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 763/1530] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:25:20.722560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad7544-db5a-4eb4-b908-d73fff0581d1 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:20.723099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad7544-db5a-4eb4-b908-d73fff0581d1 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:20.723099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad7544-db5a-4eb4-b908-d73fff0581d1 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:20.724430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:20.729526 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d81508-4d6f-46a5-9844-a58dac5121d2 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:20.729936 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d81508-4d6f-46a5-9844-a58dac5121d2 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:20.730084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d81508-4d6f-46a5-9844-a58dac5121d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:20.750584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b9d81508-4d6f-46a5-9844-a58dac5121d2 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:20.751777 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b9d81508-4d6f-46a5-9844-a58dac5121d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:20.759530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:21.053046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad7544-db5a-4eb4-b908-d73fff0581d1 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:21.063788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad7544-db5a-4eb4-b908-d73fff0581d1 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:21.067374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfe67a-47d0-4270-a484-3118dd9d9de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:21.067965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfe67a-47d0-4270-a484-3118dd9d9de4 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:21.068278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfe67a-47d0-4270-a484-3118dd9d9de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:21.068555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfe67a-47d0-4270-a484-3118dd9d9de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:21.077609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadfe67a-47d0-4270-a484-3118dd9d9de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:21.079037 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadfe67a-47d0-4270-a484-3118dd9d9de4 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:21.088222 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 769/1533] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:24.544350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17d0dc-f42d-4a05-91df-ae3d1515c34e None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:24.544886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17d0dc-f42d-4a05-91df-ae3d1515c34e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:24.544886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17d0dc-f42d-4a05-91df-ae3d1515c34e None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:24.546230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:24.551431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cb01e-6fd0-439f-a924-03951e2ff2e1 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:24.551849 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cb01e-6fd0-439f-a924-03951e2ff2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:24.551990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cb01e-6fd0-439f-a924-03951e2ff2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:24.571287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-8a1cb01e-6fd0-439f-a924-03951e2ff2e1 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:24.572926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8a1cb01e-6fd0-439f-a924-03951e2ff2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:24.581644 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:25.139214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17d0dc-f42d-4a05-91df-ae3d1515c34e None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:25.140992 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-1e17d0dc-f42d-4a05-91df-ae3d1515c34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:25.206426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:25.219394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17d0dc-f42d-4a05-91df-ae3d1515c34e None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:25.223080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c0274-925e-4738-bdf2-5ab7c346f223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:25.223722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c0274-925e-4738-bdf2-5ab7c346f223 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:25.224005 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c0274-925e-4738-bdf2-5ab7c346f223 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:25.224310 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c0274-925e-4738-bdf2-5ab7c346f223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:25.233593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c0274-925e-4738-bdf2-5ab7c346f223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:25.235010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c0274-925e-4738-bdf2-5ab7c346f223 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:25.244840 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 766/1536] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:25:27.164553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584d1ad-1214-488c-ae30-fede43d5c5b3 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:27.165169 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584d1ad-1214-488c-ae30-fede43d5c5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:27.165169 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584d1ad-1214-488c-ae30-fede43d5c5b3 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:27.166449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:27.171738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e1e6-e2bb-43be-9112-93176c99ee2a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:27.172155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e1e6-e2bb-43be-9112-93176c99ee2a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:27.172326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e1e6-e2bb-43be-9112-93176c99ee2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:27.196433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3f37e1e6-e2bb-43be-9112-93176c99ee2a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:27.197638 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3f37e1e6-e2bb-43be-9112-93176c99ee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:27.207776 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:27.762011 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584d1ad-1214-488c-ae30-fede43d5c5b3 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:27.763686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7584d1ad-1214-488c-ae30-fede43d5c5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:27.776648 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584d1ad-1214-488c-ae30-fede43d5c5b3 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:27.780637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39910ac8-0b88-4d9a-9bfa-cbd98cba02d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:27.781273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39910ac8-0b88-4d9a-9bfa-cbd98cba02d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:27.781551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39910ac8-0b88-4d9a-9bfa-cbd98cba02d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:27.781817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39910ac8-0b88-4d9a-9bfa-cbd98cba02d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:27.790470 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39910ac8-0b88-4d9a-9bfa-cbd98cba02d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:27.791880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39910ac8-0b88-4d9a-9bfa-cbd98cba02d9 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:27.801646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 772/1539] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:37.190125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=11 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:37.797581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=33 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:52.231099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86365341-b4ff-4a06-ae14-ff2975797f87 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.231772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86365341-b4ff-4a06-ae14-ff2975797f87 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.231772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86365341-b4ff-4a06-ae14-ff2975797f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.239092 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aad645-1d29-4c0e-8844-8039d2218484 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.240159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aad645-1d29-4c0e-8844-8039d2218484 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.240756 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aad645-1d29-4c0e-8844-8039d2218484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.261142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-86365341-b4ff-4a06-ae14-ff2975797f87 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:52.262393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-86365341-b4ff-4a06-ae14-ff2975797f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:52.273536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-07aad645-1d29-4c0e-8844-8039d2218484 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:52.275158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-07aad645-1d29-4c0e-8844-8039d2218484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:52.290895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 768/1540] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:52.294512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 773/1541] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:52.303744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f80c70-9bda-4a97-bb7e-f05b96178165 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.304119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f80c70-9bda-4a97-bb7e-f05b96178165 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.304339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f80c70-9bda-4a97-bb7e-f05b96178165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.305780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dce30-f00b-455a-ab6a-95e562ec2362 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.306310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dce30-f00b-455a-ab6a-95e562ec2362 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.306856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dce30-f00b-455a-ab6a-95e562ec2362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.340272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-02f80c70-9bda-4a97-bb7e-f05b96178165 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:52.340272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-02f80c70-9bda-4a97-bb7e-f05b96178165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:52.362939 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 769/1542] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:52.369270 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f80c70-9bda-4a97-bb7e-f05b96178165 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:52.375579 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-450dce30-f00b-455a-ab6a-95e562ec2362 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:52.375579 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-450dce30-f00b-455a-ab6a-95e562ec2362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:52.389948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f80c70-9bda-4a97-bb7e-f05b96178165 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:52.397483 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b67fe-a64b-4c96-95ee-0529cf85493f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:52.404104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 774/1543] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:26:52.404283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b67fe-a64b-4c96-95ee-0529cf85493f admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.404283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b67fe-a64b-4c96-95ee-0529cf85493f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.404283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b67fe-a64b-4c96-95ee-0529cf85493f admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.404283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b67fe-a64b-4c96-95ee-0529cf85493f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:26:52.404283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b67fe-a64b-4c96-95ee-0529cf85493f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:52.404283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b67fe-a64b-4c96-95ee-0529cf85493f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:52.404818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dce30-f00b-455a-ab6a-95e562ec2362 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:52.418936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dce30-f00b-455a-ab6a-95e562ec2362 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:52.422367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 770/1544] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:26:52.431480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b67fe-a64b-4c96-95ee-0529cf85493f admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:52.434610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1cef4-ae7e-4785-bd2e-08c02d5b706c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:52.435374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1cef4-ae7e-4785-bd2e-08c02d5b706c admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.435660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1cef4-ae7e-4785-bd2e-08c02d5b706c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.435953 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1cef4-ae7e-4785-bd2e-08c02d5b706c admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.436854 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1cef4-ae7e-4785-bd2e-08c02d5b706c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:26:52.440919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1cef4-ae7e-4785-bd2e-08c02d5b706c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:52.440919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: WARNING py.warnings [None req-e8b1cef4-ae7e-4785-bd2e-08c02d5b706c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:26:52.440919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: warnings.warn(msg) Aug 29 20:26:52.440919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: Aug 29 20:26:52.440919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1cef4-ae7e-4785-bd2e-08c02d5b706c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:52.443791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b67fe-a64b-4c96-95ee-0529cf85493f admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:52.464405 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 775/1545] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:26:52.467654 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203c91b-562d-4257-8c9d-de7dd78fb36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:52.468332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203c91b-562d-4257-8c9d-de7dd78fb36e admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.468612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203c91b-562d-4257-8c9d-de7dd78fb36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.468886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203c91b-562d-4257-8c9d-de7dd78fb36e admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.469488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1cef4-ae7e-4785-bd2e-08c02d5b706c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:52.469615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203c91b-562d-4257-8c9d-de7dd78fb36e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:52.470605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: WARNING py.warnings [None req-2203c91b-562d-4257-8c9d-de7dd78fb36e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:26:52.470605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: warnings.warn(msg) Aug 29 20:26:52.470605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: Aug 29 20:26:52.471811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203c91b-562d-4257-8c9d-de7dd78fb36e admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:52.487633 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1cef4-ae7e-4785-bd2e-08c02d5b706c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:52.507046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7197f-b4e7-4f59-af4a-2b1c243e54b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:52.509795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7197f-b4e7-4f59-af4a-2b1c243e54b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.509795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7197f-b4e7-4f59-af4a-2b1c243e54b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.509795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7197f-b4e7-4f59-af4a-2b1c243e54b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.509795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7197f-b4e7-4f59-af4a-2b1c243e54b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:52.511983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7197f-b4e7-4f59-af4a-2b1c243e54b3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:52.622016 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 771/1546] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:26:52.626024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203c91b-562d-4257-8c9d-de7dd78fb36e admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:52.649683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 776/1547] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/projects => generated 371 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:52.656711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7197f-b4e7-4f59-af4a-2b1c243e54b3 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:52.724879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203c91b-562d-4257-8c9d-de7dd78fb36e admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:52.751825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7197f-b4e7-4f59-af4a-2b1c243e54b3 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:52.759316 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc3bf2-1c77-42f8-95d6-b0c23d20668f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:52.760702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc3bf2-1c77-42f8-95d6-b0c23d20668f admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.761831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc3bf2-1c77-42f8-95d6-b0c23d20668f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.762321 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc3bf2-1c77-42f8-95d6-b0c23d20668f admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.763352 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc3bf2-1c77-42f8-95d6-b0c23d20668f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:52.765881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: WARNING py.warnings [None req-e0fc3bf2-1c77-42f8-95d6-b0c23d20668f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:26:52.765881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: warnings.warn(msg) Aug 29 20:26:52.765881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: Aug 29 20:26:52.768122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc3bf2-1c77-42f8-95d6-b0c23d20668f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:52.800614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba977ca-c1fd-4e35-b950-428dff4ea023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:52.801594 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba977ca-c1fd-4e35-b950-428dff4ea023 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.801725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba977ca-c1fd-4e35-b950-428dff4ea023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.802435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba977ca-c1fd-4e35-b950-428dff4ea023 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.802663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba977ca-c1fd-4e35-b950-428dff4ea023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:52.804692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba977ca-c1fd-4e35-b950-428dff4ea023 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:52.891038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 777/1548] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:26:52.892970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af1765-1d41-42f5-bf06-e673efd0e66f None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.893355 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af1765-1d41-42f5-bf06-e673efd0e66f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.893725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af1765-1d41-42f5-bf06-e673efd0e66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.914021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-77af1765-1d41-42f5-bf06-e673efd0e66f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:52.915289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-77af1765-1d41-42f5-bf06-e673efd0e66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:52.922051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 772/1549] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/users => generated 287 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:26:52.928506 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 778/1550] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:52.928635 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a6d9f-60e5-48ac-91af-efd458dd92e3 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.928735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb4668-684b-40f7-86de-2f81fe048f24 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.928830 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a6d9f-60e5-48ac-91af-efd458dd92e3 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.928955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb4668-684b-40f7-86de-2f81fe048f24 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.929066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a6d9f-60e5-48ac-91af-efd458dd92e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.929166 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb4668-684b-40f7-86de-2f81fe048f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.946170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-4cbb4668-684b-40f7-86de-2f81fe048f24 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:52.947371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4cbb4668-684b-40f7-86de-2f81fe048f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:52.950174 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-a22a6d9f-60e5-48ac-91af-efd458dd92e3 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:52.951338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a22a6d9f-60e5-48ac-91af-efd458dd92e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:52.956087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 779/1551] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:52.957108 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ca638-e6cf-4578-8efe-775b500f1fca None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.957389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ca638-e6cf-4578-8efe-775b500f1fca None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.957659 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ca638-e6cf-4578-8efe-775b500f1fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.960264 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 773/1552] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:52.961722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df375a-8ef5-4c59-835c-f2c4dcd346e0 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.962164 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df375a-8ef5-4c59-835c-f2c4dcd346e0 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.962532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df375a-8ef5-4c59-835c-f2c4dcd346e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.981001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-751ca638-e6cf-4578-8efe-775b500f1fca None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:52.982190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-70df375a-8ef5-4c59-835c-f2c4dcd346e0 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:52.982310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-751ca638-e6cf-4578-8efe-775b500f1fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:52.983335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-70df375a-8ef5-4c59-835c-f2c4dcd346e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:52.989993 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 780/1553] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:52.990749 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ca638-e6cf-4578-8efe-775b500f1fca None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:52.998901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 774/1554] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:53.000157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308843d9-adac-4376-9597-a83b0e080bbc None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.000331 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ca638-e6cf-4578-8efe-775b500f1fca None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.000751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308843d9-adac-4376-9597-a83b0e080bbc None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.001099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308843d9-adac-4376-9597-a83b0e080bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.008103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53057d-16ed-4fa1-a082-e6b26fcb09b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.008935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53057d-16ed-4fa1-a082-e6b26fcb09b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.010139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53057d-16ed-4fa1-a082-e6b26fcb09b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.010437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53057d-16ed-4fa1-a082-e6b26fcb09b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.011239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53057d-16ed-4fa1-a082-e6b26fcb09b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.012931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53057d-16ed-4fa1-a082-e6b26fcb09b7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.020500 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-308843d9-adac-4376-9597-a83b0e080bbc None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:53.021691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-308843d9-adac-4376-9597-a83b0e080bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:53.023875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 781/1555] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:53.024706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53057d-16ed-4fa1-a082-e6b26fcb09b7 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.030179 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 775/1556] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:53.031356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cb548-6fe7-4c55-839d-5b85a39ed842 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.031638 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cb548-6fe7-4c55-839d-5b85a39ed842 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.031911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cb548-6fe7-4c55-839d-5b85a39ed842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.037355 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53057d-16ed-4fa1-a082-e6b26fcb09b7 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.042358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b634d-a887-4c1c-922d-50b3356a1782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.043603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b634d-a887-4c1c-922d-50b3356a1782 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.043888 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b634d-a887-4c1c-922d-50b3356a1782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.044159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b634d-a887-4c1c-922d-50b3356a1782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.044832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b634d-a887-4c1c-922d-50b3356a1782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.046425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b634d-a887-4c1c-922d-50b3356a1782 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.051466 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-af9cb548-6fe7-4c55-839d-5b85a39ed842 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:53.052678 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-af9cb548-6fe7-4c55-839d-5b85a39ed842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:53.056282 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 782/1557] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:53.057632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8c9de-8ea5-49cd-918b-351bbe6cb900 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.057809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8c9de-8ea5-49cd-918b-351bbe6cb900 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.058222 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8c9de-8ea5-49cd-918b-351bbe6cb900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.060762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 776/1558] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:53.061929 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff6e5c-be89-4788-84f1-d155a5bf90ae None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.062211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff6e5c-be89-4788-84f1-d155a5bf90ae None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.062486 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff6e5c-be89-4788-84f1-d155a5bf90ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.081126 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-34ff6e5c-be89-4788-84f1-d155a5bf90ae None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:53.081295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-4ae8c9de-8ea5-49cd-918b-351bbe6cb900 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:53.082368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-34ff6e5c-be89-4788-84f1-d155a5bf90ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:53.082515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4ae8c9de-8ea5-49cd-918b-351bbe6cb900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:53.094247 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 777/1559] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:53.094394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 783/1560] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:53.094394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b70df-ebe9-45e7-a1ff-ab440ec5f6ea None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.094586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466cb2d-0229-4e83-969d-3bec01ed9425 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.094672 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b70df-ebe9-45e7-a1ff-ab440ec5f6ea None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.094756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466cb2d-0229-4e83-969d-3bec01ed9425 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.094852 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b70df-ebe9-45e7-a1ff-ab440ec5f6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.094941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466cb2d-0229-4e83-969d-3bec01ed9425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.108773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:53.112140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-671b70df-ebe9-45e7-a1ff-ab440ec5f6ea None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:53.113350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-671b70df-ebe9-45e7-a1ff-ab440ec5f6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:53.114958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-8466cb2d-0229-4e83-969d-3bec01ed9425 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:53.116151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8466cb2d-0229-4e83-969d-3bec01ed9425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:53.121335 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 784/1561] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:53.128544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b70df-ebe9-45e7-a1ff-ab440ec5f6ea None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.128658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 778/1562] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:53.128658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291176d4-560e-4c6b-837c-b507adb24ef9 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.128658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291176d4-560e-4c6b-837c-b507adb24ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.128658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291176d4-560e-4c6b-837c-b507adb24ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.131759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b70df-ebe9-45e7-a1ff-ab440ec5f6ea None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.136469 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13b543-ce2d-4e95-9fe7-035ba79bd5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.137076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13b543-ce2d-4e95-9fe7-035ba79bd5da admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.137359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13b543-ce2d-4e95-9fe7-035ba79bd5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.137634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13b543-ce2d-4e95-9fe7-035ba79bd5da admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.138283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13b543-ce2d-4e95-9fe7-035ba79bd5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.139934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13b543-ce2d-4e95-9fe7-035ba79bd5da admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.145774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-291176d4-560e-4c6b-837c-b507adb24ef9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:53.146966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-291176d4-560e-4c6b-837c-b507adb24ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:53.150984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 785/1563] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:53.151785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13b543-ce2d-4e95-9fe7-035ba79bd5da admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.155790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 779/1564] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:53.156613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291176d4-560e-4c6b-837c-b507adb24ef9 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.162124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13b543-ce2d-4e95-9fe7-035ba79bd5da admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.166020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291176d4-560e-4c6b-837c-b507adb24ef9 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.166593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca1cb9-24cb-4eed-a4b1-43ef089c3f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.168437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1cb9-24cb-4eed-a4b1-43ef089c3f92 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.168437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1cb9-24cb-4eed-a4b1-43ef089c3f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.168437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1cb9-24cb-4eed-a4b1-43ef089c3f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.168909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca1cb9-24cb-4eed-a4b1-43ef089c3f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.170480 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca1cb9-24cb-4eed-a4b1-43ef089c3f92 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.171344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff331a-7213-48be-84d5-9c7a84a3279d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.171921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff331a-7213-48be-84d5-9c7a84a3279d admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.172219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff331a-7213-48be-84d5-9c7a84a3279d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.172494 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff331a-7213-48be-84d5-9c7a84a3279d admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.173121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff331a-7213-48be-84d5-9c7a84a3279d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:26:53.173417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff331a-7213-48be-84d5-9c7a84a3279d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.175080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff331a-7213-48be-84d5-9c7a84a3279d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.183681 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 786/1565] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:53.183681 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca1cb9-24cb-4eed-a4b1-43ef089c3f92 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.187545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 780/1566] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:26:53.188428 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff331a-7213-48be-84d5-9c7a84a3279d admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.191228 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca1cb9-24cb-4eed-a4b1-43ef089c3f92 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.196504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d86dd-0200-41e0-ab6e-0496c9b970ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.197097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d86dd-0200-41e0-ab6e-0496c9b970ec admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.197411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d86dd-0200-41e0-ab6e-0496c9b970ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.197808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d86dd-0200-41e0-ab6e-0496c9b970ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.198451 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d86dd-0200-41e0-ab6e-0496c9b970ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:26:53.198710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff331a-7213-48be-84d5-9c7a84a3279d admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.199327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d86dd-0200-41e0-ab6e-0496c9b970ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.200847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d86dd-0200-41e0-ab6e-0496c9b970ec admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.203851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011c812-6fb1-4c05-a3bc-08fe0eaeaab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.204462 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011c812-6fb1-4c05-a3bc-08fe0eaeaab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.204740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011c812-6fb1-4c05-a3bc-08fe0eaeaab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.205033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011c812-6fb1-4c05-a3bc-08fe0eaeaab6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.205683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011c812-6fb1-4c05-a3bc-08fe0eaeaab6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:26:53.205982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011c812-6fb1-4c05-a3bc-08fe0eaeaab6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.207579 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011c812-6fb1-4c05-a3bc-08fe0eaeaab6 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.220752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 787/1567] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:26:53.220752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d86dd-0200-41e0-ab6e-0496c9b970ec admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.220985 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 781/1568] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:26:53.221331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011c812-6fb1-4c05-a3bc-08fe0eaeaab6 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.225649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d86dd-0200-41e0-ab6e-0496c9b970ec admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.231401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c0b91-10a6-4542-89be-f08575e1d055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.232022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c0b91-10a6-4542-89be-f08575e1d055 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.232358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c0b91-10a6-4542-89be-f08575e1d055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.232484 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011c812-6fb1-4c05-a3bc-08fe0eaeaab6 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.232977 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c0b91-10a6-4542-89be-f08575e1d055 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.233298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c0b91-10a6-4542-89be-f08575e1d055 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:26:53.233597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c0b91-10a6-4542-89be-f08575e1d055 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.235161 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c0b91-10a6-4542-89be-f08575e1d055 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.237307 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbdabe-8f42-40f9-b017-036d566d4e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.238339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbdabe-8f42-40f9-b017-036d566d4e08 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.238617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbdabe-8f42-40f9-b017-036d566d4e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.238888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbdabe-8f42-40f9-b017-036d566d4e08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.239525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbdabe-8f42-40f9-b017-036d566d4e08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:26:53.239827 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbdabe-8f42-40f9-b017-036d566d4e08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.241430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbdabe-8f42-40f9-b017-036d566d4e08 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.241873 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:53.246846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 788/1569] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:26:53.247585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c0b91-10a6-4542-89be-f08575e1d055 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.254865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 782/1570] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:26:53.254865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbdabe-8f42-40f9-b017-036d566d4e08 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.258783 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c0b91-10a6-4542-89be-f08575e1d055 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.262201 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43152f-ddf6-4861-8418-3a616da69b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.262870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43152f-ddf6-4861-8418-3a616da69b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.263141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43152f-ddf6-4861-8418-3a616da69b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.263407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43152f-ddf6-4861-8418-3a616da69b47 admin admin] PATH_INFO: `/v3/projects/8cf35ea597fc452ab2999e230751a26a/users/eafa589bad64499795ff5401ad26c8a8/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.264980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbdabe-8f42-40f9-b017-036d566d4e08 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.270496 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c2a19-d3b1-4c2c-8f95-efa8723bbe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.271371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c2a19-d3b1-4c2c-8f95-efa8723bbe25 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.271738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c2a19-d3b1-4c2c-8f95-efa8723bbe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.272082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c2a19-d3b1-4c2c-8f95-efa8723bbe25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.272813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c2a19-d3b1-4c2c-8f95-efa8723bbe25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:26:53.273197 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c2a19-d3b1-4c2c-8f95-efa8723bbe25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.274781 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c2a19-d3b1-4c2c-8f95-efa8723bbe25 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.279237 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43152f-ddf6-4861-8418-3a616da69b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cf35ea597fc452ab2999e230751a26a, user_id=eafa589bad64499795ff5401ad26c8a8, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.282037 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43152f-ddf6-4861-8418-3a616da69b47 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.287542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 783/1571] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:26:53.288618 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c2a19-d3b1-4c2c-8f95-efa8723bbe25 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.298942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c2a19-d3b1-4c2c-8f95-efa8723bbe25 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.304293 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf106d-8369-4eb6-9968-ef3949a0b1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.304420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf106d-8369-4eb6-9968-ef3949a0b1bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.304700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf106d-8369-4eb6-9968-ef3949a0b1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.304971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf106d-8369-4eb6-9968-ef3949a0b1bb admin admin] PATH_INFO: `/v3/projects/b01211b781474f90934dafe9263ee0a5/users/d57d38e5168049c0beb124cbab043a60/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.320801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf106d-8369-4eb6-9968-ef3949a0b1bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01211b781474f90934dafe9263ee0a5, user_id=d57d38e5168049c0beb124cbab043a60, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.321829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: WARNING py.warnings [None req-44bf106d-8369-4eb6-9968-ef3949a0b1bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:26:53.321829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: warnings.warn(msg) Aug 29 20:26:53.321829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: Aug 29 20:26:53.323769 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf106d-8369-4eb6-9968-ef3949a0b1bb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.365879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 789/1572] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:53 2022] PUT /identity/v3/projects/8cf35ea597fc452ab2999e230751a26a/users/eafa589bad64499795ff5401ad26c8a8/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:53.366676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43152f-ddf6-4861-8418-3a616da69b47 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.395093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 784/1573] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:53 2022] PUT /identity/v3/projects/b01211b781474f90934dafe9263ee0a5/users/d57d38e5168049c0beb124cbab043a60/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:53.396238 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf106d-8369-4eb6-9968-ef3949a0b1bb admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.430564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43152f-ddf6-4861-8418-3a616da69b47 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.458188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf106d-8369-4eb6-9968-ef3949a0b1bb admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.462945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9ed4f-6057-4f99-a3e3-7cd53061abf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.463549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9ed4f-6057-4f99-a3e3-7cd53061abf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.463831 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9ed4f-6057-4f99-a3e3-7cd53061abf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.464106 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9ed4f-6057-4f99-a3e3-7cd53061abf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.464804 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9ed4f-6057-4f99-a3e3-7cd53061abf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.466701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9ed4f-6057-4f99-a3e3-7cd53061abf7 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.488948 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883bfa5-de8e-4200-8fdf-76c80bf904c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.489553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883bfa5-de8e-4200-8fdf-76c80bf904c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.489950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883bfa5-de8e-4200-8fdf-76c80bf904c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.490232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883bfa5-de8e-4200-8fdf-76c80bf904c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.490862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883bfa5-de8e-4200-8fdf-76c80bf904c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.492813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883bfa5-de8e-4200-8fdf-76c80bf904c8 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.576541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 785/1574] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:53.576541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9ed4f-6057-4f99-a3e3-7cd53061abf7 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.602232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 790/1575] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/projects => generated 383 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:53.603738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883bfa5-de8e-4200-8fdf-76c80bf904c8 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.644965 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9ed4f-6057-4f99-a3e3-7cd53061abf7 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.649684 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:53.669208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883bfa5-de8e-4200-8fdf-76c80bf904c8 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.673915 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e0259-4177-4ed1-9219-458dd1dfbd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.674527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e0259-4177-4ed1-9219-458dd1dfbd0a admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.674808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e0259-4177-4ed1-9219-458dd1dfbd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.675084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e0259-4177-4ed1-9219-458dd1dfbd0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.675715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e0259-4177-4ed1-9219-458dd1dfbd0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.677648 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e0259-4177-4ed1-9219-458dd1dfbd0a admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.689658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:53.702317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61753a2d-1afd-4a6f-86cc-41e86e40e0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.702941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61753a2d-1afd-4a6f-86cc-41e86e40e0be admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.703226 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61753a2d-1afd-4a6f-86cc-41e86e40e0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.703502 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61753a2d-1afd-4a6f-86cc-41e86e40e0be admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.704136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61753a2d-1afd-4a6f-86cc-41e86e40e0be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.706114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61753a2d-1afd-4a6f-86cc-41e86e40e0be admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.796141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 786/1576] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/projects => generated 405 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:53.799934 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61753a2d-1afd-4a6f-86cc-41e86e40e0be admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.800074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 791/1577] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/projects => generated 379 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:53.800074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e0259-4177-4ed1-9219-458dd1dfbd0a admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:53.858656 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e0259-4177-4ed1-9219-458dd1dfbd0a admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.859524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61753a2d-1afd-4a6f-86cc-41e86e40e0be admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:53.863448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67132ca5-fcfc-4638-bf72-79aa020fee57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.864048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132ca5-fcfc-4638-bf72-79aa020fee57 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.864365 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132ca5-fcfc-4638-bf72-79aa020fee57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.864530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d7b21-b6b4-4d8e-8b5d-c026532cb246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:53.864650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132ca5-fcfc-4638-bf72-79aa020fee57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.865151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d7b21-b6b4-4d8e-8b5d-c026532cb246 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.865362 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67132ca5-fcfc-4638-bf72-79aa020fee57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.865466 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d7b21-b6b4-4d8e-8b5d-c026532cb246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.865707 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d7b21-b6b4-4d8e-8b5d-c026532cb246 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.866791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d7b21-b6b4-4d8e-8b5d-c026532cb246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:53.867281 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67132ca5-fcfc-4638-bf72-79aa020fee57 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.868789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d7b21-b6b4-4d8e-8b5d-c026532cb246 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:53.967665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 792/1578] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:53.968944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846394b-bb93-4fbf-b65e-f3ef081cfb9a None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.969229 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846394b-bb93-4fbf-b65e-f3ef081cfb9a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.969503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846394b-bb93-4fbf-b65e-f3ef081cfb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.990786 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 787/1579] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/projects => generated 393 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:53.999205 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965f782-6364-48c4-8a7d-abcb95088cf2 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:53.999205 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965f782-6364-48c4-8a7d-abcb95088cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:53.999205 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965f782-6364-48c4-8a7d-abcb95088cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:53.999637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-b846394b-bb93-4fbf-b65e-f3ef081cfb9a None None] MFA Rules not processed for user `eafa589bad64499795ff5401ad26c8a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:54.002270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b846394b-bb93-4fbf-b65e-f3ef081cfb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:54.023925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b965f782-6364-48c4-8a7d-abcb95088cf2 None None] MFA Rules not processed for user `d57d38e5168049c0beb124cbab043a60`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:54.025157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b965f782-6364-48c4-8a7d-abcb95088cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:54.066359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 793/1580] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:54.067135 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846394b-bb93-4fbf-b65e-f3ef081cfb9a None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.090152 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 788/1581] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:54.090981 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965f782-6364-48c4-8a7d-abcb95088cf2 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.100117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:54.137592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846394b-bb93-4fbf-b65e-f3ef081cfb9a None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.142502 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cb9f9-98c7-40eb-a41b-b7cabda405eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.143107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cb9f9-98c7-40eb-a41b-b7cabda405eb admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.143388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cb9f9-98c7-40eb-a41b-b7cabda405eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.143658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cb9f9-98c7-40eb-a41b-b7cabda405eb admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.144316 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8cb9f9-98c7-40eb-a41b-b7cabda405eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.146282 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8cb9f9-98c7-40eb-a41b-b7cabda405eb admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.154861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965f782-6364-48c4-8a7d-abcb95088cf2 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.157693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:54.160395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe2a0c-a72c-484c-bbc3-b3f83aa38c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.161000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe2a0c-a72c-484c-bbc3-b3f83aa38c10 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.161281 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe2a0c-a72c-484c-bbc3-b3f83aa38c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.161553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe2a0c-a72c-484c-bbc3-b3f83aa38c10 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.162192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe2a0c-a72c-484c-bbc3-b3f83aa38c10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.164123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe2a0c-a72c-484c-bbc3-b3f83aa38c10 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.235728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 794/1582] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:26:54 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:54.236629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cb9f9-98c7-40eb-a41b-b7cabda405eb admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.246615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cb9f9-98c7-40eb-a41b-b7cabda405eb admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.251528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462323e-4f67-477d-966f-646c30e882f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.252300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462323e-4f67-477d-966f-646c30e882f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.252516 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462323e-4f67-477d-966f-646c30e882f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.252822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462323e-4f67-477d-966f-646c30e882f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.253446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462323e-4f67-477d-966f-646c30e882f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.255475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462323e-4f67-477d-966f-646c30e882f7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.258350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 789/1583] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:26:54 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:54.261572 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe2a0c-a72c-484c-bbc3-b3f83aa38c10 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.269102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe2a0c-a72c-484c-bbc3-b3f83aa38c10 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.273634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0276e-7952-4e3c-bfdd-a10646e5366f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.274207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0276e-7952-4e3c-bfdd-a10646e5366f admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.274482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0276e-7952-4e3c-bfdd-a10646e5366f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.274753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0276e-7952-4e3c-bfdd-a10646e5366f admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.275357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0276e-7952-4e3c-bfdd-a10646e5366f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.277286 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0276e-7952-4e3c-bfdd-a10646e5366f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.394979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 795/1584] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:26:54 2022] POST /identity/v3/users => generated 303 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:54.395153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 790/1584] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:26:54 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:54.395750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0276e-7952-4e3c-bfdd-a10646e5366f admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.395856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462323e-4f67-477d-966f-646c30e882f7 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.405369 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0276e-7952-4e3c-bfdd-a10646e5366f admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.405648 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462323e-4f67-477d-966f-646c30e882f7 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.410028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84685f1-e07f-42a8-87f0-719d395fc0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.410189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712425b-13c9-4415-9084-d8f6130f8a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.410582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84685f1-e07f-42a8-87f0-719d395fc0aa admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.410783 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712425b-13c9-4415-9084-d8f6130f8a80 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.410891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84685f1-e07f-42a8-87f0-719d395fc0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.411091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712425b-13c9-4415-9084-d8f6130f8a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.411363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712425b-13c9-4415-9084-d8f6130f8a80 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.411531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84685f1-e07f-42a8-87f0-719d395fc0aa admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.411976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712425b-13c9-4415-9084-d8f6130f8a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.412180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84685f1-e07f-42a8-87f0-719d395fc0aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.413906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712425b-13c9-4415-9084-d8f6130f8a80 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.414157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84685f1-e07f-42a8-87f0-719d395fc0aa admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.520149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 796/1585] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:26:54 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:54.521204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712425b-13c9-4415-9084-d8f6130f8a80 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.531146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712425b-13c9-4415-9084-d8f6130f8a80 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.532553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 791/1586] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:26:54 2022] POST /identity/v3/users => generated 297 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:54.533799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd029e-c4c5-47ea-9ab7-9442dd60aaaa None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.534083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd029e-c4c5-47ea-9ab7-9442dd60aaaa None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.534361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd029e-c4c5-47ea-9ab7-9442dd60aaaa None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.536057 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 792/1587] 10.222.0.85 () {54 vars in 962 bytes} [Mon Aug 29 20:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:54.536543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd029e-c4c5-47ea-9ab7-9442dd60aaaa None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.539222 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b3ec2-b205-4e37-9468-ecfb1791a340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.539826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b3ec2-b205-4e37-9468-ecfb1791a340 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.540113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b3ec2-b205-4e37-9468-ecfb1791a340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.540496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b3ec2-b205-4e37-9468-ecfb1791a340 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.541170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b3ec2-b205-4e37-9468-ecfb1791a340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.543123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b3ec2-b205-4e37-9468-ecfb1791a340 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.548840 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd029e-c4c5-47ea-9ab7-9442dd60aaaa None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.553465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2141b5-fd89-4847-87b5-4767e8ea5bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.554072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2141b5-fd89-4847-87b5-4767e8ea5bbb admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.563576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2141b5-fd89-4847-87b5-4767e8ea5bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.563576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2141b5-fd89-4847-87b5-4767e8ea5bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.563576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2141b5-fd89-4847-87b5-4767e8ea5bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.563576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2141b5-fd89-4847-87b5-4767e8ea5bbb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.567226 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 793/1588] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:54.568453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2141b5-fd89-4847-87b5-4767e8ea5bbb admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.578254 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2141b5-fd89-4847-87b5-4767e8ea5bbb admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.583067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a19e2-3a77-411e-bf66-f190759e540e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.583684 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a19e2-3a77-411e-bf66-f190759e540e admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.583979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a19e2-3a77-411e-bf66-f190759e540e admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.584263 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a19e2-3a77-411e-bf66-f190759e540e admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.584918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a19e2-3a77-411e-bf66-f190759e540e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.586527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a19e2-3a77-411e-bf66-f190759e540e admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.596056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 794/1589] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:54.603639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a19e2-3a77-411e-bf66-f190759e540e admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.605962 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a19e2-3a77-411e-bf66-f190759e540e admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.610538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1a80e-9160-4c1e-a7d4-92d5f52e3abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.611112 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1a80e-9160-4c1e-a7d4-92d5f52e3abc admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.611387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1a80e-9160-4c1e-a7d4-92d5f52e3abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.611656 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1a80e-9160-4c1e-a7d4-92d5f52e3abc admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.612306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc1a80e-9160-4c1e-a7d4-92d5f52e3abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.613858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc1a80e-9160-4c1e-a7d4-92d5f52e3abc admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.624922 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 795/1590] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:54.625848 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1a80e-9160-4c1e-a7d4-92d5f52e3abc admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.635043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1a80e-9160-4c1e-a7d4-92d5f52e3abc admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.640428 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bfcc4-95ff-4f57-a0f0-db3ef7a2fa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.643694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bfcc4-95ff-4f57-a0f0-db3ef7a2fa6b admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.643694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bfcc4-95ff-4f57-a0f0-db3ef7a2fa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.643694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bfcc4-95ff-4f57-a0f0-db3ef7a2fa6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.643694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bfcc4-95ff-4f57-a0f0-db3ef7a2fa6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.644064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bfcc4-95ff-4f57-a0f0-db3ef7a2fa6b admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.653832 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 796/1591] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:54.654730 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bfcc4-95ff-4f57-a0f0-db3ef7a2fa6b admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.663767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bfcc4-95ff-4f57-a0f0-db3ef7a2fa6b admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.668330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f64f52-7f94-42f7-8165-ad5a87fef09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.668867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f64f52-7f94-42f7-8165-ad5a87fef09c admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.669144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f64f52-7f94-42f7-8165-ad5a87fef09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.669413 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f64f52-7f94-42f7-8165-ad5a87fef09c admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.670011 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f64f52-7f94-42f7-8165-ad5a87fef09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.671509 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f64f52-7f94-42f7-8165-ad5a87fef09c admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.680818 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 797/1592] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:54.681677 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f64f52-7f94-42f7-8165-ad5a87fef09c admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.691099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f64f52-7f94-42f7-8165-ad5a87fef09c admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.695506 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4336cfc-846a-4f1e-878b-03d7caf80f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.696092 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4336cfc-846a-4f1e-878b-03d7caf80f94 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.696398 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4336cfc-846a-4f1e-878b-03d7caf80f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.696668 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4336cfc-846a-4f1e-878b-03d7caf80f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.697275 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4336cfc-846a-4f1e-878b-03d7caf80f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.698758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4336cfc-846a-4f1e-878b-03d7caf80f94 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.706114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 797/1593] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:26:54 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:54.707376 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec21e6-97ba-48d8-a8e6-78b6af8bd308 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.707663 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec21e6-97ba-48d8-a8e6-78b6af8bd308 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.707938 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec21e6-97ba-48d8-a8e6-78b6af8bd308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.712122 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 798/1594] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:54.712122 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4336cfc-846a-4f1e-878b-03d7caf80f94 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.719764 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4336cfc-846a-4f1e-878b-03d7caf80f94 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.745373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-dbec21e6-97ba-48d8-a8e6-78b6af8bd308 None None] MFA Rules not processed for user `805df3047fee45c7a702d72a5c082eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:54.747150 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-dbec21e6-97ba-48d8-a8e6-78b6af8bd308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:54.772292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:54.778224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78f9b3-8b65-41c4-9483-2167b4396b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.778826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78f9b3-8b65-41c4-9483-2167b4396b36 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.779111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78f9b3-8b65-41c4-9483-2167b4396b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.779382 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78f9b3-8b65-41c4-9483-2167b4396b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.780026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78f9b3-8b65-41c4-9483-2167b4396b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.781635 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78f9b3-8b65-41c4-9483-2167b4396b36 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.795616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 799/1595] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:54.795616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78f9b3-8b65-41c4-9483-2167b4396b36 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.803163 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78f9b3-8b65-41c4-9483-2167b4396b36 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.807890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5811df-f0dc-4711-aaef-a4748543f9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.808557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5811df-f0dc-4711-aaef-a4748543f9da admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.808839 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5811df-f0dc-4711-aaef-a4748543f9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.809622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5811df-f0dc-4711-aaef-a4748543f9da admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.810282 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5811df-f0dc-4711-aaef-a4748543f9da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.811878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5811df-f0dc-4711-aaef-a4748543f9da admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.815380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 798/1596] 10.222.0.85 () {58 vars in 1059 bytes} [Mon Aug 29 20:26:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:54.816241 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec21e6-97ba-48d8-a8e6-78b6af8bd308 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.824691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 800/1597] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:54.824691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5811df-f0dc-4711-aaef-a4748543f9da admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.829694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec21e6-97ba-48d8-a8e6-78b6af8bd308 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.832015 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5811df-f0dc-4711-aaef-a4748543f9da admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.845906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079f879-02aa-447f-b924-7e117dabb33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.845906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079f879-02aa-447f-b924-7e117dabb33d admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.845906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079f879-02aa-447f-b924-7e117dabb33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.845906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079f879-02aa-447f-b924-7e117dabb33d admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.846319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079f879-02aa-447f-b924-7e117dabb33d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.846416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335eb57-74bf-471b-ae15-3a8b570773f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.846416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335eb57-74bf-471b-ae15-3a8b570773f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.846416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335eb57-74bf-471b-ae15-3a8b570773f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.846416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335eb57-74bf-471b-ae15-3a8b570773f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.846780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079f879-02aa-447f-b924-7e117dabb33d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.846869 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335eb57-74bf-471b-ae15-3a8b570773f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.846869 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335eb57-74bf-471b-ae15-3a8b570773f2 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.848151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 799/1598] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:54.848974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079f879-02aa-447f-b924-7e117dabb33d admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.853522 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 801/1599] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:54.853522 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335eb57-74bf-471b-ae15-3a8b570773f2 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.859308 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079f879-02aa-447f-b924-7e117dabb33d admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.863568 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335eb57-74bf-471b-ae15-3a8b570773f2 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.864279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f29dc-e4df-4e52-b7c0-73175f884d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.864871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f29dc-e4df-4e52-b7c0-73175f884d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.865147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f29dc-e4df-4e52-b7c0-73175f884d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.865414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f29dc-e4df-4e52-b7c0-73175f884d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.866045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f29dc-e4df-4e52-b7c0-73175f884d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.867621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f29dc-e4df-4e52-b7c0-73175f884d45 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.868364 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e52f3b-5551-44f0-b0ff-cb44a3fbacb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.869049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e52f3b-5551-44f0-b0ff-cb44a3fbacb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.869327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e52f3b-5551-44f0-b0ff-cb44a3fbacb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.869609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e52f3b-5551-44f0-b0ff-cb44a3fbacb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.870249 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e52f3b-5551-44f0-b0ff-cb44a3fbacb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.872720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e52f3b-5551-44f0-b0ff-cb44a3fbacb7 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.878752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 800/1600] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:54.879607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f29dc-e4df-4e52-b7c0-73175f884d45 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.889382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f29dc-e4df-4e52-b7c0-73175f884d45 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.894478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c87ac-02f6-4e79-a36f-01e245599119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.903112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c87ac-02f6-4e79-a36f-01e245599119 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.903112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c87ac-02f6-4e79-a36f-01e245599119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.903112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c87ac-02f6-4e79-a36f-01e245599119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.903112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c87ac-02f6-4e79-a36f-01e245599119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.903112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c87ac-02f6-4e79-a36f-01e245599119 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.907456 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 801/1601] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:54.908138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c87ac-02f6-4e79-a36f-01e245599119 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:54.917199 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c87ac-02f6-4e79-a36f-01e245599119 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:54.922234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e421c8-a2a9-4488-a303-257a06ba724e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:54.922841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e421c8-a2a9-4488-a303-257a06ba724e admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.923123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e421c8-a2a9-4488-a303-257a06ba724e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.923403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e421c8-a2a9-4488-a303-257a06ba724e admin admin] PATH_INFO: `/v3/projects/0eaee9c0d7014c88b9b08f4f5e020a66/users/c823ba2f1a1b441989d27846a43d60ad/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.938034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e421c8-a2a9-4488-a303-257a06ba724e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eaee9c0d7014c88b9b08f4f5e020a66, user_id=c823ba2f1a1b441989d27846a43d60ad, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:54.940685 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e421c8-a2a9-4488-a303-257a06ba724e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:54.997998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 802/1603] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:26:54 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:54.997998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e52f3b-5551-44f0-b0ff-cb44a3fbacb7 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:55.007981 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e52f3b-5551-44f0-b0ff-cb44a3fbacb7 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:55.013981 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a91a81-6d1f-4091-b49a-c7398e938a25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:55.014561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a91a81-6d1f-4091-b49a-c7398e938a25 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.014848 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a91a81-6d1f-4091-b49a-c7398e938a25 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.015122 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a91a81-6d1f-4091-b49a-c7398e938a25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.068793 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a91a81-6d1f-4091-b49a-c7398e938a25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:55.070378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a91a81-6d1f-4091-b49a-c7398e938a25 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:55.079681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 803/1604] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:55.080462 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a91a81-6d1f-4091-b49a-c7398e938a25 service nova] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:55.089811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a91a81-6d1f-4091-b49a-c7398e938a25 service nova] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:55.094359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583cb46-ebb6-4f89-89ab-c26d28832a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:55.094968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583cb46-ebb6-4f89-89ab-c26d28832a62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.095251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583cb46-ebb6-4f89-89ab-c26d28832a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.095533 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583cb46-ebb6-4f89-89ab-c26d28832a62 admin admin] PATH_INFO: `/v3/projects/fab91f53f7d0452189b5977d835ffb24/users/2311207200ea41169272e4097aa85c38/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.109580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583cb46-ebb6-4f89-89ab-c26d28832a62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab91f53f7d0452189b5977d835ffb24, user_id=2311207200ea41169272e4097aa85c38, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:55.112205 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583cb46-ebb6-4f89-89ab-c26d28832a62 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:55.138538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 802/1605] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:54 2022] PUT /identity/v3/projects/0eaee9c0d7014c88b9b08f4f5e020a66/users/c823ba2f1a1b441989d27846a43d60ad/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:55.142862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e421c8-a2a9-4488-a303-257a06ba724e admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:55.159814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:55.201684 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e421c8-a2a9-4488-a303-257a06ba724e admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:55.206749 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dab12b-2b2a-4316-9755-6760a2d95c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:55.207608 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dab12b-2b2a-4316-9755-6760a2d95c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.208058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dab12b-2b2a-4316-9755-6760a2d95c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.208651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dab12b-2b2a-4316-9755-6760a2d95c76 admin admin] PATH_INFO: `/v3/projects/8412896b98ed427a9668ec022c5656e5/users/e13627d7433d4b1cb494e0221b55d6b6/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.212449 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 804/1606] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:55 2022] PUT /identity/v3/projects/fab91f53f7d0452189b5977d835ffb24/users/2311207200ea41169272e4097aa85c38/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:55.212902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583cb46-ebb6-4f89-89ab-c26d28832a62 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:55.225219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dab12b-2b2a-4316-9755-6760a2d95c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8412896b98ed427a9668ec022c5656e5, user_id=e13627d7433d4b1cb494e0221b55d6b6, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:55.228117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dab12b-2b2a-4316-9755-6760a2d95c76 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:55.283884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583cb46-ebb6-4f89-89ab-c26d28832a62 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:55.289128 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb952d5-5351-4010-9afe-e3f134d2a72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:55.289790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb952d5-5351-4010-9afe-e3f134d2a72b admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.290069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb952d5-5351-4010-9afe-e3f134d2a72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.290349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb952d5-5351-4010-9afe-e3f134d2a72b admin admin] PATH_INFO: `/v3/projects/bad47a00811647e39f59ff4e0b6bd4ce/users/83fa185f035945e8a4943d325b943261/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.304725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb952d5-5351-4010-9afe-e3f134d2a72b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad47a00811647e39f59ff4e0b6bd4ce, user_id=83fa185f035945e8a4943d325b943261, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:55.307353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb952d5-5351-4010-9afe-e3f134d2a72b admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:55.321497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 803/1607] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:55 2022] PUT /identity/v3/projects/8412896b98ed427a9668ec022c5656e5/users/e13627d7433d4b1cb494e0221b55d6b6/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:55.325009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dab12b-2b2a-4316-9755-6760a2d95c76 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:55.381216 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dab12b-2b2a-4316-9755-6760a2d95c76 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:55.385963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299fd45-8f3c-43bc-9769-efad00d551dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:55.387992 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299fd45-8f3c-43bc-9769-efad00d551dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.387992 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299fd45-8f3c-43bc-9769-efad00d551dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.387992 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299fd45-8f3c-43bc-9769-efad00d551dd admin admin] PATH_INFO: `/v3/projects/6adb6c50ed6c4fa5beebf037da1e73e7/users/3f4b05a527e14d2599e03077c6c5f96f/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.390899 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 805/1608] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:55 2022] PUT /identity/v3/projects/bad47a00811647e39f59ff4e0b6bd4ce/users/83fa185f035945e8a4943d325b943261/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:55.391971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb952d5-5351-4010-9afe-e3f134d2a72b admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:55.403798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299fd45-8f3c-43bc-9769-efad00d551dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6adb6c50ed6c4fa5beebf037da1e73e7, user_id=3f4b05a527e14d2599e03077c6c5f96f, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:55.406614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299fd45-8f3c-43bc-9769-efad00d551dd admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:55.407516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:55.460528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb952d5-5351-4010-9afe-e3f134d2a72b admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:55.460528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa520a3-f0e0-4a89-b287-6deabe9df8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:55.460528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa520a3-f0e0-4a89-b287-6deabe9df8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.460528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa520a3-f0e0-4a89-b287-6deabe9df8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.461133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa520a3-f0e0-4a89-b287-6deabe9df8a5 admin admin] PATH_INFO: `/v3/projects/66be3e70cbf742269fe9306a7b7e8fe0/users/336345e5437341ba9859d776c44ea4be/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.474795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa520a3-f0e0-4a89-b287-6deabe9df8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66be3e70cbf742269fe9306a7b7e8fe0, user_id=336345e5437341ba9859d776c44ea4be, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:55.477513 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa520a3-f0e0-4a89-b287-6deabe9df8a5 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:55.514076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 804/1609] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:55 2022] PUT /identity/v3/projects/6adb6c50ed6c4fa5beebf037da1e73e7/users/3f4b05a527e14d2599e03077c6c5f96f/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:55.514902 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299fd45-8f3c-43bc-9769-efad00d551dd admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:55.560696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 806/1610] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:55 2022] PUT /identity/v3/projects/66be3e70cbf742269fe9306a7b7e8fe0/users/336345e5437341ba9859d776c44ea4be/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:55.561970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19e104-a26a-4657-9788-a1ff23f20e5d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.562284 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19e104-a26a-4657-9788-a1ff23f20e5d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.562613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19e104-a26a-4657-9788-a1ff23f20e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.581769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299fd45-8f3c-43bc-9769-efad00d551dd admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:55.582782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-4a19e104-a26a-4657-9788-a1ff23f20e5d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:55.583963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-4a19e104-a26a-4657-9788-a1ff23f20e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:55.614769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:55.644604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7d7d7-d631-4b6c-955a-6bd2f0717099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:55.645245 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7d7d7-d631-4b6c-955a-6bd2f0717099 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.645538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7d7d7-d631-4b6c-955a-6bd2f0717099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.645825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7d7d7-d631-4b6c-955a-6bd2f0717099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.646513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7d7d7-d631-4b6c-955a-6bd2f0717099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:55.647305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 807/1611] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:55.650841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7d7d7-d631-4b6c-955a-6bd2f0717099 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:55.650958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccce8fb-e9ee-4718-8337-e2ad94a9808b None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.650958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccce8fb-e9ee-4718-8337-e2ad94a9808b None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.650958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccce8fb-e9ee-4718-8337-e2ad94a9808b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.665172 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 805/1612] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:55.666778 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb9b0c-271e-456c-b6c6-3c337188b1fd None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.667170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb9b0c-271e-456c-b6c6-3c337188b1fd None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.668050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb9b0c-271e-456c-b6c6-3c337188b1fd None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.669790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 806/1613] 10.222.0.85 () {54 vars in 912 bytes} [Mon Aug 29 20:26:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:55.670852 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6ff8-2fc5-45b1-af90-5b3f4cf2a3d1 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.671197 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6ff8-2fc5-45b1-af90-5b3f4cf2a3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.671678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6ff8-2fc5-45b1-af90-5b3f4cf2a3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.681893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-8ccce8fb-e9ee-4718-8337-e2ad94a9808b None None] MFA Rules not processed for user `2311207200ea41169272e4097aa85c38`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:55.683152 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8ccce8fb-e9ee-4718-8337-e2ad94a9808b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:55.701230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-d4cf6ff8-2fc5-45b1-af90-5b3f4cf2a3d1 None None] MFA Rules not processed for user `e13627d7433d4b1cb494e0221b55d6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:55.702552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-d4cf6ff8-2fc5-45b1-af90-5b3f4cf2a3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:55.752574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 808/1614] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:55.753752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfdd3b5-d62a-411e-9e37-8471c7937cb0 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.754042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfdd3b5-d62a-411e-9e37-8471c7937cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.754321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfdd3b5-d62a-411e-9e37-8471c7937cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.769137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 807/1615] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:55 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:55.769972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcab01-d0ca-4c81-b56a-32128f72e13d None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.770490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcab01-d0ca-4c81-b56a-32128f72e13d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.770878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcab01-d0ca-4c81-b56a-32128f72e13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.791431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-82fcab01-d0ca-4c81-b56a-32128f72e13d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:55.792831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-82fcab01-d0ca-4c81-b56a-32128f72e13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:55.796059 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-8dfdd3b5-d62a-411e-9e37-8471c7937cb0 None None] MFA Rules not processed for user `83fa185f035945e8a4943d325b943261`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:55.798191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8dfdd3b5-d62a-411e-9e37-8471c7937cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:55.800666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 808/1616] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:55.805298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b5915-8415-4a67-a7b5-fa2c7ed0b1cd None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.805682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b5915-8415-4a67-a7b5-fa2c7ed0b1cd None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.806063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b5915-8415-4a67-a7b5-fa2c7ed0b1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.810674 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:55.825857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-535b5915-8415-4a67-a7b5-fa2c7ed0b1cd None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:55.831859 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-535b5915-8415-4a67-a7b5-fa2c7ed0b1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:55.834981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 809/1617] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:55.835524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b5915-8415-4a67-a7b5-fa2c7ed0b1cd None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:55.866423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 809/1618] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:55.867336 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfdd3b5-d62a-411e-9e37-8471c7937cb0 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:55.877383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfdd3b5-d62a-411e-9e37-8471c7937cb0 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:55.882051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fca452-c663-43c2-a035-244cd86d195d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:55.882726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fca452-c663-43c2-a035-244cd86d195d admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.886733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fca452-c663-43c2-a035-244cd86d195d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.886733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fca452-c663-43c2-a035-244cd86d195d admin admin] PATH_INFO: `/v3/projects/c7a4412c22ea47ea90d48bc754986e71/users/621c9c15b8ee4fbd8493c0f21dd57e21/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.897299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b5915-8415-4a67-a7b5-fa2c7ed0b1cd None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:55.898407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fca452-c663-43c2-a035-244cd86d195d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a4412c22ea47ea90d48bc754986e71, user_id=621c9c15b8ee4fbd8493c0f21dd57e21, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:55.901043 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667747fd-cc9b-4088-b674-01c0a7ba782a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:55.901201 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fca452-c663-43c2-a035-244cd86d195d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:55.902410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667747fd-cc9b-4088-b674-01c0a7ba782a service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.902776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667747fd-cc9b-4088-b674-01c0a7ba782a service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.903162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667747fd-cc9b-4088-b674-01c0a7ba782a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.912655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667747fd-cc9b-4088-b674-01c0a7ba782a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:55.914226 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667747fd-cc9b-4088-b674-01c0a7ba782a service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:55.924268 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 810/1619] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:55.925662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16935d06-0df7-40ea-b17c-1daccae0b2d3 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.925949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16935d06-0df7-40ea-b17c-1daccae0b2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.926221 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16935d06-0df7-40ea-b17c-1daccae0b2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.945784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-16935d06-0df7-40ea-b17c-1daccae0b2d3 None None] MFA Rules not processed for user `805df3047fee45c7a702d72a5c082eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:55.946970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-16935d06-0df7-40ea-b17c-1daccae0b2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:55.977196 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 810/1620] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:55 2022] PUT /identity/v3/projects/c7a4412c22ea47ea90d48bc754986e71/users/621c9c15b8ee4fbd8493c0f21dd57e21/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:55.982018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fca452-c663-43c2-a035-244cd86d195d admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:55.996292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:56.019587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 811/1621] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:26:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:56.020651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec01bb-f372-4f33-8c75-33ed29621dfe None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.020938 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec01bb-f372-4f33-8c75-33ed29621dfe None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.021213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec01bb-f372-4f33-8c75-33ed29621dfe None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.022596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 812/1622] 10.222.0.85 () {54 vars in 962 bytes} [Mon Aug 29 20:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:56.023296 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec01bb-f372-4f33-8c75-33ed29621dfe None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:56.024938 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-87ec01bb-f372-4f33-8c75-33ed29621dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:56.041502 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fca452-c663-43c2-a035-244cd86d195d admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:56.046718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24518f-609e-4e4c-b320-4f27a2f0ff5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:56.047318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24518f-609e-4e4c-b320-4f27a2f0ff5d admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.047599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24518f-609e-4e4c-b320-4f27a2f0ff5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.047870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24518f-609e-4e4c-b320-4f27a2f0ff5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.048530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24518f-609e-4e4c-b320-4f27a2f0ff5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:56.050488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24518f-609e-4e4c-b320-4f27a2f0ff5d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:56.083758 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec01bb-f372-4f33-8c75-33ed29621dfe None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:56.087627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bff21-323a-40f5-89ae-215d46c34e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:56.088272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bff21-323a-40f5-89ae-215d46c34e39 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.088562 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bff21-323a-40f5-89ae-215d46c34e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.088837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bff21-323a-40f5-89ae-215d46c34e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.098272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200bff21-323a-40f5-89ae-215d46c34e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:56.099724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200bff21-323a-40f5-89ae-215d46c34e39 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:56.109895 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 813/1623] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:56.110784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bff21-323a-40f5-89ae-215d46c34e39 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:56.118441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bff21-323a-40f5-89ae-215d46c34e39 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:56.122259 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dd9ab-5517-4f0e-9da1-fdda845a5199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:56.122852 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd9ab-5517-4f0e-9da1-fdda845a5199 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.123136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd9ab-5517-4f0e-9da1-fdda845a5199 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.123459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd9ab-5517-4f0e-9da1-fdda845a5199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.135361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 811/1624] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:26:55 2022] POST /identity/v3/projects => generated 401 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:56.136120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3dd9ab-5517-4f0e-9da1-fdda845a5199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:56.136333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24518f-609e-4e4c-b320-4f27a2f0ff5d admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:56.138006 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3dd9ab-5517-4f0e-9da1-fdda845a5199 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:56.149967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 814/1625] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:56.150747 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dd9ab-5517-4f0e-9da1-fdda845a5199 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:56.160582 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dd9ab-5517-4f0e-9da1-fdda845a5199 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:56.208400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24518f-609e-4e4c-b320-4f27a2f0ff5d admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:56.213051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27887c21-57b7-4a7e-bf7c-82fba1970497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:56.213652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27887c21-57b7-4a7e-bf7c-82fba1970497 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.213934 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27887c21-57b7-4a7e-bf7c-82fba1970497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.214201 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27887c21-57b7-4a7e-bf7c-82fba1970497 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.214934 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27887c21-57b7-4a7e-bf7c-82fba1970497 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:56.216914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27887c21-57b7-4a7e-bf7c-82fba1970497 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:56.222468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7da76-b046-4d6f-bdad-3def026abdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:56.223171 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7da76-b046-4d6f-bdad-3def026abdbc admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.223549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7da76-b046-4d6f-bdad-3def026abdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.224027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7da76-b046-4d6f-bdad-3def026abdbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.224798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7da76-b046-4d6f-bdad-3def026abdbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:56.227078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7da76-b046-4d6f-bdad-3def026abdbc admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:56.240319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 815/1626] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:56.241704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e9b26-545f-4ac3-b8c6-332d339d845e None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.241996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e9b26-545f-4ac3-b8c6-332d339d845e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.242266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e9b26-545f-4ac3-b8c6-332d339d845e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.263201 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-a11e9b26-545f-4ac3-b8c6-332d339d845e None None] MFA Rules not processed for user `805df3047fee45c7a702d72a5c082eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:56.265374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a11e9b26-545f-4ac3-b8c6-332d339d845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:56.328756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:56.336126 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 812/1627] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:26:56 2022] POST /identity/v3/projects => generated 381 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:56.337195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27887c21-57b7-4a7e-bf7c-82fba1970497 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:56.340680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:56.355736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 816/1628] 10.222.0.85 () {58 vars in 1059 bytes} [Mon Aug 29 20:26:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:56.356686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e9b26-545f-4ac3-b8c6-332d339d845e None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:56.369161 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e9b26-545f-4ac3-b8c6-332d339d845e None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:56.404091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27887c21-57b7-4a7e-bf7c-82fba1970497 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:56.409733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416503b-0c33-4caf-be97-f0d1a12e922c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:56.410594 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416503b-0c33-4caf-be97-f0d1a12e922c service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.410918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416503b-0c33-4caf-be97-f0d1a12e922c service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.411305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416503b-0c33-4caf-be97-f0d1a12e922c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.443353 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475e4ae-12d9-4fc1-9f54-38f6c3339cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:56.444091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475e4ae-12d9-4fc1-9f54-38f6c3339cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.444485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475e4ae-12d9-4fc1-9f54-38f6c3339cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.445187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475e4ae-12d9-4fc1-9f54-38f6c3339cf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.445596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475e4ae-12d9-4fc1-9f54-38f6c3339cf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:56.447389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475e4ae-12d9-4fc1-9f54-38f6c3339cf5 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:56.481632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416503b-0c33-4caf-be97-f0d1a12e922c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:56.483072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416503b-0c33-4caf-be97-f0d1a12e922c service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:56.493620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 813/1629] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:56.494377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416503b-0c33-4caf-be97-f0d1a12e922c service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:56.504767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416503b-0c33-4caf-be97-f0d1a12e922c service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:56.509406 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1ea08-816a-41eb-a7e2-7339040ad4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:56.510064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1ea08-816a-41eb-a7e2-7339040ad4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.510348 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1ea08-816a-41eb-a7e2-7339040ad4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.517889 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1ea08-816a-41eb-a7e2-7339040ad4f5 admin admin] PATH_INFO: `/v3/projects/c7a4412c22ea47ea90d48bc754986e71/users/621c9c15b8ee4fbd8493c0f21dd57e21/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.517889 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1ea08-816a-41eb-a7e2-7339040ad4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a4412c22ea47ea90d48bc754986e71, user_id=621c9c15b8ee4fbd8493c0f21dd57e21, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:56.517889 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1ea08-816a-41eb-a7e2-7339040ad4f5 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:56.529419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 814/1630] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:56 2022] PUT /identity/v3/projects/c7a4412c22ea47ea90d48bc754986e71/users/621c9c15b8ee4fbd8493c0f21dd57e21/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:56.537273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1ea08-816a-41eb-a7e2-7339040ad4f5 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:56.553394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 817/1631] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:26:56 2022] POST /identity/v3/users => generated 307 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:56.559030 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475e4ae-12d9-4fc1-9f54-38f6c3339cf5 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:56.578356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475e4ae-12d9-4fc1-9f54-38f6c3339cf5 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:56.666618 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1ea08-816a-41eb-a7e2-7339040ad4f5 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:56.691343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb66eaa-1d2e-4820-a591-f29d57c6fd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:56.691343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb66eaa-1d2e-4820-a591-f29d57c6fd9f admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.691343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb66eaa-1d2e-4820-a591-f29d57c6fd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.691343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb66eaa-1d2e-4820-a591-f29d57c6fd9f admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.692709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb66eaa-1d2e-4820-a591-f29d57c6fd9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:56.695669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52009486-0c40-4b26-be05-5889d8b88397 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:56.695977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb66eaa-1d2e-4820-a591-f29d57c6fd9f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:56.696556 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52009486-0c40-4b26-be05-5889d8b88397 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.696931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52009486-0c40-4b26-be05-5889d8b88397 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.697355 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52009486-0c40-4b26-be05-5889d8b88397 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.782760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52009486-0c40-4b26-be05-5889d8b88397 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:56.785059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52009486-0c40-4b26-be05-5889d8b88397 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:56.798021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 818/1632] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:56.800906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e09757-65dd-4c16-8c88-657743a05105 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.806397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e09757-65dd-4c16-8c88-657743a05105 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.806397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e09757-65dd-4c16-8c88-657743a05105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.843316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 815/1633] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:26:56 2022] POST /identity/v3/users => generated 297 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:26:56.846613 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-a9e09757-65dd-4c16-8c88-657743a05105 None None] MFA Rules not processed for user `621c9c15b8ee4fbd8493c0f21dd57e21`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:56.846733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb66eaa-1d2e-4820-a591-f29d57c6fd9f admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:56.846834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a9e09757-65dd-4c16-8c88-657743a05105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:56.865872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:56.869086 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb66eaa-1d2e-4820-a591-f29d57c6fd9f admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:56.878572 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc54e3-9978-4999-af5a-35829fe07104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:56.880632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc54e3-9978-4999-af5a-35829fe07104 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.881083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc54e3-9978-4999-af5a-35829fe07104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.881731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc54e3-9978-4999-af5a-35829fe07104 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.883710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc54e3-9978-4999-af5a-35829fe07104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:56.886849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc54e3-9978-4999-af5a-35829fe07104 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:56.904806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 816/1634] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:56.907235 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc54e3-9978-4999-af5a-35829fe07104 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:56.921346 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc54e3-9978-4999-af5a-35829fe07104 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:56.932670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21dd2e-2fbb-40a6-810d-9bfc869e9a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:56.933722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21dd2e-2fbb-40a6-810d-9bfc869e9a60 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:56.935395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21dd2e-2fbb-40a6-810d-9bfc869e9a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:56.935674 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21dd2e-2fbb-40a6-810d-9bfc869e9a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:56.936430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21dd2e-2fbb-40a6-810d-9bfc869e9a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:56.939091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21dd2e-2fbb-40a6-810d-9bfc869e9a60 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:56.941569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:56.970334 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 819/1635] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:56.970540 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 817/1636] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:56.970657 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e09757-65dd-4c16-8c88-657743a05105 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:56.972206 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21dd2e-2fbb-40a6-810d-9bfc869e9a60 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:56.995143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21dd2e-2fbb-40a6-810d-9bfc869e9a60 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:57.005977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703a001-c42e-4ad0-bc5a-b9f5d6d4a371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:57.006694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703a001-c42e-4ad0-bc5a-b9f5d6d4a371 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:57.012746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703a001-c42e-4ad0-bc5a-b9f5d6d4a371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:57.012746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703a001-c42e-4ad0-bc5a-b9f5d6d4a371 admin admin] PATH_INFO: `/v3/projects/057454e7c1b74b2fb61ce703b1484f84/users/581c78daeb264cf4be5b0269c3301db9/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:57.017086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e09757-65dd-4c16-8c88-657743a05105 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:57.027868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ebe3a-2268-4489-a910-4507383b2366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:57.028655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ebe3a-2268-4489-a910-4507383b2366 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:57.028993 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ebe3a-2268-4489-a910-4507383b2366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:57.029324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ebe3a-2268-4489-a910-4507383b2366 admin admin] PATH_INFO: `/v3/projects/0eed4aebbf1c464280885f2fe18d4975/users/92d54b99717c482da47783077ee860f0/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:57.052347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ebe3a-2268-4489-a910-4507383b2366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eed4aebbf1c464280885f2fe18d4975, user_id=92d54b99717c482da47783077ee860f0, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:57.053390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703a001-c42e-4ad0-bc5a-b9f5d6d4a371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=057454e7c1b74b2fb61ce703b1484f84, user_id=581c78daeb264cf4be5b0269c3301db9, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:57.055981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ebe3a-2268-4489-a910-4507383b2366 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:57.057767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703a001-c42e-4ad0-bc5a-b9f5d6d4a371 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:57.197131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 820/1637] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:56 2022] PUT /identity/v3/projects/0eed4aebbf1c464280885f2fe18d4975/users/92d54b99717c482da47783077ee860f0/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:57.197131 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ebe3a-2268-4489-a910-4507383b2366 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:57.197394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 818/1638] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:56 2022] PUT /identity/v3/projects/057454e7c1b74b2fb61ce703b1484f84/users/581c78daeb264cf4be5b0269c3301db9/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:57.197394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce195e-b15e-4f7d-b9dc-1313b3aa5aed None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:57.197394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce195e-b15e-4f7d-b9dc-1313b3aa5aed None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:57.197394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce195e-b15e-4f7d-b9dc-1313b3aa5aed None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:57.197394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 819/1639] 10.222.0.85 () {54 vars in 912 bytes} [Mon Aug 29 20:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:57.198197 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce195e-b15e-4f7d-b9dc-1313b3aa5aed None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:57.282929 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce195e-b15e-4f7d-b9dc-1313b3aa5aed None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:57.288801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbd9ce-4973-44ea-8974-d45d172bbe1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:57.289416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbd9ce-4973-44ea-8974-d45d172bbe1a admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:57.289696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbd9ce-4973-44ea-8974-d45d172bbe1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:57.289972 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbd9ce-4973-44ea-8974-d45d172bbe1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:57.290737 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbd9ce-4973-44ea-8974-d45d172bbe1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:57.293074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbd9ce-4973-44ea-8974-d45d172bbe1a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:57.300451 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ebe3a-2268-4489-a910-4507383b2366 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:57.306141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 820/1640] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:57.307086 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbd9ce-4973-44ea-8974-d45d172bbe1a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:57.318098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbd9ce-4973-44ea-8974-d45d172bbe1a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:57.324676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927bbfe3-e303-4f5e-b316-0501c28c9000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:57.328926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bbfe3-e303-4f5e-b316-0501c28c9000 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:57.330017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bbfe3-e303-4f5e-b316-0501c28c9000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:57.330506 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bbfe3-e303-4f5e-b316-0501c28c9000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:57.340396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927bbfe3-e303-4f5e-b316-0501c28c9000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:57.340396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927bbfe3-e303-4f5e-b316-0501c28c9000 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:57.353071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 821/1641] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:26:57.354721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb320c-5d12-45be-8781-99e4c1244ea8 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:57.355966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb320c-5d12-45be-8781-99e4c1244ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:57.357499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb320c-5d12-45be-8781-99e4c1244ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:57.373422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73abe4-119d-4d68-af04-ef56204109fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:57.374122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73abe4-119d-4d68-af04-ef56204109fc service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:57.374496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73abe4-119d-4d68-af04-ef56204109fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:57.374767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73abe4-119d-4d68-af04-ef56204109fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:57.386963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3dbb320c-5d12-45be-8781-99e4c1244ea8 None None] MFA Rules not processed for user `805df3047fee45c7a702d72a5c082eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:57.389127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3dbb320c-5d12-45be-8781-99e4c1244ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:57.470878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:57.481878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73abe4-119d-4d68-af04-ef56204109fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:57.483484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73abe4-119d-4d68-af04-ef56204109fc service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:57.490980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 822/1642] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:26:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:57.497858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb320c-5d12-45be-8781-99e4c1244ea8 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:57.514650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 821/1643] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:57.515534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb320c-5d12-45be-8781-99e4c1244ea8 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:57.520147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73abe4-119d-4d68-af04-ef56204109fc service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:57.524503 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462f29c-d92d-421c-b848-5996df6d6a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:57.525624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462f29c-d92d-421c-b848-5996df6d6a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:57.525997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462f29c-d92d-421c-b848-5996df6d6a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:57.526366 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462f29c-d92d-421c-b848-5996df6d6a86 admin admin] PATH_INFO: `/v3/projects/0eed4aebbf1c464280885f2fe18d4975/users/92d54b99717c482da47783077ee860f0/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:57.529762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462f29c-d92d-421c-b848-5996df6d6a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eed4aebbf1c464280885f2fe18d4975, user_id=92d54b99717c482da47783077ee860f0, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:57.533521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462f29c-d92d-421c-b848-5996df6d6a86 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:57.541558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73abe4-119d-4d68-af04-ef56204109fc service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:57.555044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 823/1644] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:57 2022] PUT /identity/v3/projects/0eed4aebbf1c464280885f2fe18d4975/users/92d54b99717c482da47783077ee860f0/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:57.567438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830604b2-c1fa-49e0-89b8-1b37cb911a40 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:57.567438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830604b2-c1fa-49e0-89b8-1b37cb911a40 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:57.567438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830604b2-c1fa-49e0-89b8-1b37cb911a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:57.597041 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-830604b2-c1fa-49e0-89b8-1b37cb911a40 None None] MFA Rules not processed for user `92d54b99717c482da47783077ee860f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:57.598409 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-830604b2-c1fa-49e0-89b8-1b37cb911a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:57.635395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:57.650669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57878553-a376-4f48-a585-87c1172a0879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:57.651364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57878553-a376-4f48-a585-87c1172a0879 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:57.651712 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57878553-a376-4f48-a585-87c1172a0879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:57.653450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57878553-a376-4f48-a585-87c1172a0879 admin admin] PATH_INFO: `/v3/projects/057454e7c1b74b2fb61ce703b1484f84/users/581c78daeb264cf4be5b0269c3301db9/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:57.672587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57878553-a376-4f48-a585-87c1172a0879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=057454e7c1b74b2fb61ce703b1484f84, user_id=581c78daeb264cf4be5b0269c3301db9, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:57.675332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57878553-a376-4f48-a585-87c1172a0879 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:57.729558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 822/1645] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:26:57 2022] PUT /identity/v3/projects/057454e7c1b74b2fb61ce703b1484f84/users/581c78daeb264cf4be5b0269c3301db9/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:26:57.729558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57878553-a376-4f48-a585-87c1172a0879 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:57.757188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 824/1646] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:57 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:57.759410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04353892-6733-4c76-ab15-77828fb79285 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:58.262853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04353892-6733-4c76-ab15-77828fb79285 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:58.262853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04353892-6733-4c76-ab15-77828fb79285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:58.262853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-04353892-6733-4c76-ab15-77828fb79285 None None] MFA Rules not processed for user `581c78daeb264cf4be5b0269c3301db9`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:58.262853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-04353892-6733-4c76-ab15-77828fb79285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:58.267901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57878553-a376-4f48-a585-87c1172a0879 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:58.267901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27a036-1861-4d7e-8c22-701e14efca0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:58.267901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27a036-1861-4d7e-8c22-701e14efca0e service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:58.267901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27a036-1861-4d7e-8c22-701e14efca0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:58.281478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27a036-1861-4d7e-8c22-701e14efca0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:58.281478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27a036-1861-4d7e-8c22-701e14efca0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:58.282072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 825/1647] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:26:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:58.282544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27a036-1861-4d7e-8c22-701e14efca0e service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:58.282544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 823/1648] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:58.283031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04353892-6733-4c76-ab15-77828fb79285 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:58.283031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04353892-6733-4c76-ab15-77828fb79285 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:58.283031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bbb65-97f6-4f26-bcfe-ff14d34eeb80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:58.283031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bbb65-97f6-4f26-bcfe-ff14d34eeb80 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:58.283556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bbb65-97f6-4f26-bcfe-ff14d34eeb80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:58.283556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bbb65-97f6-4f26-bcfe-ff14d34eeb80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:58.312375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bbb65-97f6-4f26-bcfe-ff14d34eeb80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:58.315164 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bbb65-97f6-4f26-bcfe-ff14d34eeb80 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:58.325449 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 826/1649] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:58 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:58.450243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27a036-1861-4d7e-8c22-701e14efca0e service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:58.453235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27a036-1861-4d7e-8c22-701e14efca0e service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:58.457252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072ef0c-d438-4841-9b26-16c2c6ba0cf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:58.457915 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072ef0c-d438-4841-9b26-16c2c6ba0cf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:58.458198 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072ef0c-d438-4841-9b26-16c2c6ba0cf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:58.458476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072ef0c-d438-4841-9b26-16c2c6ba0cf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:58.468727 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072ef0c-d438-4841-9b26-16c2c6ba0cf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:58.470285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072ef0c-d438-4841-9b26-16c2c6ba0cf9 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:58.484255 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 824/1650] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:58.629225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bbb65-97f6-4f26-bcfe-ff14d34eeb80 service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:58.637901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bbb65-97f6-4f26-bcfe-ff14d34eeb80 service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:58.641921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8968ed7-14e4-4a0d-a040-99f9f67b902e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:58.642621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968ed7-14e4-4a0d-a040-99f9f67b902e service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:58.642799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968ed7-14e4-4a0d-a040-99f9f67b902e service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:58.643077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968ed7-14e4-4a0d-a040-99f9f67b902e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:58.706826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8968ed7-14e4-4a0d-a040-99f9f67b902e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:58.708896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8968ed7-14e4-4a0d-a040-99f9f67b902e service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:58.722424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 827/1651] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:59.338248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:59.432816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072ef0c-d438-4841-9b26-16c2c6ba0cf9 service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:59.447021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072ef0c-d438-4841-9b26-16c2c6ba0cf9 service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:59.452857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77b099-d6da-47a4-a797-7ebe8fd50e84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:59.454380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77b099-d6da-47a4-a797-7ebe8fd50e84 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:59.454485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77b099-d6da-47a4-a797-7ebe8fd50e84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:59.454731 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77b099-d6da-47a4-a797-7ebe8fd50e84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:59.477887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:59.544086 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8968ed7-14e4-4a0d-a040-99f9f67b902e service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:59.592940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77b099-d6da-47a4-a797-7ebe8fd50e84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:59.595008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77b099-d6da-47a4-a797-7ebe8fd50e84 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:59.618433 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 825/1652] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:59.733442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8968ed7-14e4-4a0d-a040-99f9f67b902e service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:59.746107 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:59.757913 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:59.758408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:59.758757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] PATH_INFO: `/v3/users/e13627d7433d4b1cb494e0221b55d6b6` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:59.762802 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e13627d7433d4b1cb494e0221b55d6b6)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:59.766724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: WARNING py.warnings [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:26:59.766724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: warnings.warn(msg) Aug 29 20:26:59.766724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: Aug 29 20:26:59.770087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:00.300474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e13627d7433d4b1cb494e0221b55d6b6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:00.330430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e13627d7433d4b1cb494e0221b55d6b6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:00.422386 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e13627d7433d4b1cb494e0221b55d6b6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:00.425125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e13627d7433d4b1cb494e0221b55d6b6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:00.500485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e13627d7433d4b1cb494e0221b55d6b6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:00.561071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 828/1653] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:26:59 2022] DELETE /identity/v3/users/e13627d7433d4b1cb494e0221b55d6b6 => generated 0 bytes in 1017 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:00.565928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc593d2-ee6b-4a1c-b42f-dfca46360e31 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:00.567078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc593d2-ee6b-4a1c-b42f-dfca46360e31 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:00.567867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc593d2-ee6b-4a1c-b42f-dfca46360e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:00.617326 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-5fc593d2-ee6b-4a1c-b42f-dfca46360e31 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:00.619217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5fc593d2-ee6b-4a1c-b42f-dfca46360e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:00.834733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 826/1654] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:00.869138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:00.879339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:01.032368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97f331-2035-4eb0-be50-e8e72726f2e2 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:01.040867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d62739-3555-4520-bc44-7df371190540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:01.041678 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62739-3555-4520-bc44-7df371190540 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:01.042142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62739-3555-4520-bc44-7df371190540 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:01.042455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62739-3555-4520-bc44-7df371190540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:01.071867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d62739-3555-4520-bc44-7df371190540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:01.073635 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d62739-3555-4520-bc44-7df371190540 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:01.090198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:01.104046 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 829/1655] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:02.748890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc593d2-ee6b-4a1c-b42f-dfca46360e31 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:02.751247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5fc593d2-ee6b-4a1c-b42f-dfca46360e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:02.774782 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc593d2-ee6b-4a1c-b42f-dfca46360e31 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:02.795884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:02.796567 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:02.796853 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:02.797128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] PATH_INFO: `/v3/projects/8412896b98ed427a9668ec022c5656e5` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:02.798650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8412896b98ed427a9668ec022c5656e5)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:02.799686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: WARNING py.warnings [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:27:02.799686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: warnings.warn(msg) Aug 29 20:27:02.799686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: Aug 29 20:27:02.800929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:02.988093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d62739-3555-4520-bc44-7df371190540 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:02.991095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-98d62739-3555-4520-bc44-7df371190540 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:03.008102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d62739-3555-4520-bc44-7df371190540 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:03.014373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7e474-8361-439e-9e33-350b4c945c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:03.015429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7e474-8361-439e-9e33-350b4c945c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:03.015897 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7e474-8361-439e-9e33-350b4c945c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:03.016948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7e474-8361-439e-9e33-350b4c945c67 admin admin] PATH_INFO: `/v3/users/d57d38e5168049c0beb124cbab043a60` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:03.018709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7e474-8361-439e-9e33-350b4c945c67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d57d38e5168049c0beb124cbab043a60)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:03.021433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7e474-8361-439e-9e33-350b4c945c67 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:03.181707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8412896b98ed427a9668ec022c5656e5', 'request_id': 'req-69473efc-f3c5-4547-8e80-1a5d52be1020'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:03.202260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8412896b98ed427a9668ec022c5656e5', 'request_id': 'req-69473efc-f3c5-4547-8e80-1a5d52be1020'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:03.358435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8412896b98ed427a9668ec022c5656e5', 'request_id': 'req-69473efc-f3c5-4547-8e80-1a5d52be1020'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:03.387367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fad7e474-8361-439e-9e33-350b4c945c67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd57d38e5168049c0beb124cbab043a60'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:03.409362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fad7e474-8361-439e-9e33-350b4c945c67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd57d38e5168049c0beb124cbab043a60'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:03.416439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] The token cache is being invalidate because project 8412896b98ed427a9668ec022c5656e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:03.417467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:03.420557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:03.424733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 827/1656] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:02 2022] DELETE /identity/v3/projects/8412896b98ed427a9668ec022c5656e5 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:03.431105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:03.435048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:03.522966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fad7e474-8361-439e-9e33-350b4c945c67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd57d38e5168049c0beb124cbab043a60'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:03.524602 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fad7e474-8361-439e-9e33-350b4c945c67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd57d38e5168049c0beb124cbab043a60'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:03.555475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:03.560764 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fad7e474-8361-439e-9e33-350b4c945c67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd57d38e5168049c0beb124cbab043a60'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:03.585751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 830/1657] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:02 2022] DELETE /identity/v3/users/d57d38e5168049c0beb124cbab043a60 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:03.590802 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb32c79-d7b6-4fea-b257-941b030f8b56 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:03.591250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb32c79-d7b6-4fea-b257-941b030f8b56 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:03.593930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb32c79-d7b6-4fea-b257-941b030f8b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:03.639746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69473efc-f3c5-4547-8e80-1a5d52be1020 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:03.671788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-eeb32c79-d7b6-4fea-b257-941b030f8b56 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:03.676217 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-eeb32c79-d7b6-4fea-b257-941b030f8b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:03.763854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 831/1658] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:03.780464 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb32c79-d7b6-4fea-b257-941b030f8b56 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:03.781659 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414f1af-6577-41f6-8bde-511dde642561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:03.782323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414f1af-6577-41f6-8bde-511dde642561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:03.782595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414f1af-6577-41f6-8bde-511dde642561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:03.782954 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414f1af-6577-41f6-8bde-511dde642561 admin admin] PATH_INFO: `/v3/users/92d54b99717c482da47783077ee860f0` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:03.783818 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-eeb32c79-d7b6-4fea-b257-941b030f8b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:03.784538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414f1af-6577-41f6-8bde-511dde642561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92d54b99717c482da47783077ee860f0)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:03.786465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414f1af-6577-41f6-8bde-511dde642561 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:03.919238 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb32c79-d7b6-4fea-b257-941b030f8b56 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:03.921133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5b9d3-4574-483a-9587-408cd9406450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:03.921770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5b9d3-4574-483a-9587-408cd9406450 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:03.922407 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5b9d3-4574-483a-9587-408cd9406450 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:03.922684 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5b9d3-4574-483a-9587-408cd9406450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:03.936069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5b9d3-4574-483a-9587-408cd9406450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:03.938542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5b9d3-4574-483a-9587-408cd9406450 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:03.952300 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 832/1659] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:04.154750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0414f1af-6577-41f6-8bde-511dde642561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92d54b99717c482da47783077ee860f0'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:04.156579 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0414f1af-6577-41f6-8bde-511dde642561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92d54b99717c482da47783077ee860f0'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:04.167200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0414f1af-6577-41f6-8bde-511dde642561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92d54b99717c482da47783077ee860f0'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:04.229256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0414f1af-6577-41f6-8bde-511dde642561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92d54b99717c482da47783077ee860f0'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:04.231442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0414f1af-6577-41f6-8bde-511dde642561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92d54b99717c482da47783077ee860f0'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:04.277485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 828/1660] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:03 2022] DELETE /identity/v3/users/92d54b99717c482da47783077ee860f0 => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:04.838168 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5b9d3-4574-483a-9587-408cd9406450 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:04.841512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c5a5b9d3-4574-483a-9587-408cd9406450 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:04.855885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5b9d3-4574-483a-9587-408cd9406450 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:04.888587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:04.967504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:04.968567 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:04.969024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:04.969495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] PATH_INFO: `/v3/projects/b01211b781474f90934dafe9263ee0a5` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:04.973702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01211b781474f90934dafe9263ee0a5)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:04.986864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:05.227720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a779d8-81cb-4c53-9051-cd92ad801193 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:05.228739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a779d8-81cb-4c53-9051-cd92ad801193 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:05.229238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a779d8-81cb-4c53-9051-cd92ad801193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:05.270771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01211b781474f90934dafe9263ee0a5', 'request_id': 'req-404b6e19-14f3-4b8d-93bf-a031c67b262c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:05.290447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01211b781474f90934dafe9263ee0a5', 'request_id': 'req-404b6e19-14f3-4b8d-93bf-a031c67b262c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:05.310318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-27a779d8-81cb-4c53-9051-cd92ad801193 None None] MFA Rules not processed for user `c823ba2f1a1b441989d27846a43d60ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:05.311664 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01211b781474f90934dafe9263ee0a5', 'request_id': 'req-404b6e19-14f3-4b8d-93bf-a031c67b262c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:05.317087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-27a779d8-81cb-4c53-9051-cd92ad801193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:05.446527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] The token cache is being invalidate because project b01211b781474f90934dafe9263ee0a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:05.446974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:05.447746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:05.450165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 833/1661] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:04 2022] DELETE /identity/v3/projects/b01211b781474f90934dafe9263ee0a5 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:05.452581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:05.468431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:05.525561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 829/1662] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:05.526898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca468d0c-07ed-4407-acde-0e097698d75b None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:05.527186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca468d0c-07ed-4407-acde-0e097698d75b None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:05.527458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca468d0c-07ed-4407-acde-0e097698d75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:05.596204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-ca468d0c-07ed-4407-acde-0e097698d75b None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:05.608253 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ca468d0c-07ed-4407-acde-0e097698d75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:05.700150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b6e19-14f3-4b8d-93bf-a031c67b262c admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:05.722114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:05.722785 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:05.723070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:05.723344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] PATH_INFO: `/v3/projects/0eed4aebbf1c464280885f2fe18d4975` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:05.727094 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eed4aebbf1c464280885f2fe18d4975)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:05.729577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:05.767291 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 830/1663] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:05.767610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabc6be-e172-4c61-9f71-095df25bfffd None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:05.769364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabc6be-e172-4c61-9f71-095df25bfffd None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:05.769364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabc6be-e172-4c61-9f71-095df25bfffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:05.810995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-4eabc6be-e172-4c61-9f71-095df25bfffd None None] MFA Rules not processed for user `c823ba2f1a1b441989d27846a43d60ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:05.812292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4eabc6be-e172-4c61-9f71-095df25bfffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:05.836213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 831/1664] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:05.837895 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84136a-5ca0-4043-88b6-bc9234704533 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:05.838193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84136a-5ca0-4043-88b6-bc9234704533 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:05.838496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84136a-5ca0-4043-88b6-bc9234704533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:05.861744 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-ea84136a-5ca0-4043-88b6-bc9234704533 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:05.862792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ea84136a-5ca0-4043-88b6-bc9234704533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:05.872628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 832/1665] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:05.873443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84136a-5ca0-4043-88b6-bc9234704533 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:05.874825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ea84136a-5ca0-4043-88b6-bc9234704533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:05.933441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eed4aebbf1c464280885f2fe18d4975', 'request_id': 'req-5097a952-2682-4a7e-ae5a-aac4722ff836'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:05.953721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eed4aebbf1c464280885f2fe18d4975', 'request_id': 'req-5097a952-2682-4a7e-ae5a-aac4722ff836'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:05.966780 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eed4aebbf1c464280885f2fe18d4975', 'request_id': 'req-5097a952-2682-4a7e-ae5a-aac4722ff836'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:06.023760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84136a-5ca0-4043-88b6-bc9234704533 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:06.116637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] The token cache is being invalidate because project 0eed4aebbf1c464280885f2fe18d4975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:06.123113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:06.123113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:06.123113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 834/1666] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:05 2022] DELETE /identity/v3/projects/0eed4aebbf1c464280885f2fe18d4975 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:06.123113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:06.125404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:06.197204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5f95b-463c-4ff8-8285-663a5cf52673 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:06.197902 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5f95b-463c-4ff8-8285-663a5cf52673 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:06.198106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5f95b-463c-4ff8-8285-663a5cf52673 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:06.198376 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5f95b-463c-4ff8-8285-663a5cf52673 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:06.204022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9ed5f95b-463c-4ff8-8285-663a5cf52673 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:06.332051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:06.368592 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097a952-2682-4a7e-ae5a-aac4722ff836 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:06.377176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba9677-8b5e-40b1-84cb-3258b1f4167e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:06.377815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba9677-8b5e-40b1-84cb-3258b1f4167e admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:06.378099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba9677-8b5e-40b1-84cb-3258b1f4167e admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:06.378378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba9677-8b5e-40b1-84cb-3258b1f4167e admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:06.379051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba9677-8b5e-40b1-84cb-3258b1f4167e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:06.379359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba9677-8b5e-40b1-84cb-3258b1f4167e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:06.382357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba9677-8b5e-40b1-84cb-3258b1f4167e admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:06.406661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5f95b-463c-4ff8-8285-663a5cf52673 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:06.408438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5f95b-463c-4ff8-8285-663a5cf52673 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:06.429406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 833/1667] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:06.433337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b65d6-e516-4f41-aad6-6f1762014c3b None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:06.433935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b65d6-e516-4f41-aad6-6f1762014c3b None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:06.434239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b65d6-e516-4f41-aad6-6f1762014c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:06.458143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 835/1668] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:06.469382 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba9677-8b5e-40b1-84cb-3258b1f4167e admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:06.484312 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-881b65d6-e516-4f41-aad6-6f1762014c3b None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:06.487358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-881b65d6-e516-4f41-aad6-6f1762014c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:06.506410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba9677-8b5e-40b1-84cb-3258b1f4167e admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:06.509100 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 834/1669] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:06.519980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff031873-1200-485d-863f-6acb38bef141 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:06.520285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff031873-1200-485d-863f-6acb38bef141 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:06.522800 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff031873-1200-485d-863f-6acb38bef141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:06.524801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78d356-1f5f-4c2e-b523-3ca42ff0eda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:06.531410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78d356-1f5f-4c2e-b523-3ca42ff0eda0 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:06.531410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78d356-1f5f-4c2e-b523-3ca42ff0eda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:06.531410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78d356-1f5f-4c2e-b523-3ca42ff0eda0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:06.531410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78d356-1f5f-4c2e-b523-3ca42ff0eda0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:06.531827 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78d356-1f5f-4c2e-b523-3ca42ff0eda0 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:06.573366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-ff031873-1200-485d-863f-6acb38bef141 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:06.574638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ff031873-1200-485d-863f-6acb38bef141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:06.618300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 835/1670] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:06.619662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff031873-1200-485d-863f-6acb38bef141 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:06.661927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff031873-1200-485d-863f-6acb38bef141 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:06.678902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 836/1671] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:06 2022] POST /identity/v3/projects => generated 371 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:06.683653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d8e0f-cd64-4587-a032-e170f53c649e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:06.683834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78d356-1f5f-4c2e-b523-3ca42ff0eda0 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:06.684380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d8e0f-cd64-4587-a032-e170f53c649e admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:06.684622 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d8e0f-cd64-4587-a032-e170f53c649e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:06.684907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d8e0f-cd64-4587-a032-e170f53c649e admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:06.689059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d8e0f-cd64-4587-a032-e170f53c649e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:06.689216 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d8e0f-cd64-4587-a032-e170f53c649e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:06.690844 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d8e0f-cd64-4587-a032-e170f53c649e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:06.755127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 836/1672] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:06.755127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d8e0f-cd64-4587-a032-e170f53c649e admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:06.785094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d8e0f-cd64-4587-a032-e170f53c649e admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:06.895598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78d356-1f5f-4c2e-b523-3ca42ff0eda0 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:06.908655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eee124-af98-40a0-8e0b-56e9538a9a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:06.909289 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eee124-af98-40a0-8e0b-56e9538a9a15 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:06.909574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eee124-af98-40a0-8e0b-56e9538a9a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:06.909849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eee124-af98-40a0-8e0b-56e9538a9a15 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:06.915067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eee124-af98-40a0-8e0b-56e9538a9a15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:06.915067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eee124-af98-40a0-8e0b-56e9538a9a15 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:06.948154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4d095-aa03-4512-b260-33cf8b55c2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:06.949681 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d095-aa03-4512-b260-33cf8b55c2be admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:06.949970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d095-aa03-4512-b260-33cf8b55c2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:06.950248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d095-aa03-4512-b260-33cf8b55c2be admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:06.951588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4d095-aa03-4512-b260-33cf8b55c2be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:06.953718 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4d095-aa03-4512-b260-33cf8b55c2be admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:07.098450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 837/1673] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:06 2022] POST /identity/v3/users => generated 286 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:07.103525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eee124-af98-40a0-8e0b-56e9538a9a15 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:07.126269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eee124-af98-40a0-8e0b-56e9538a9a15 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:07.140094 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d490e-65be-4b8b-9ad7-9af99c55bff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:07.141147 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d490e-65be-4b8b-9ad7-9af99c55bff6 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:07.141435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d490e-65be-4b8b-9ad7-9af99c55bff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:07.141977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d490e-65be-4b8b-9ad7-9af99c55bff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:07.142672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d490e-65be-4b8b-9ad7-9af99c55bff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:07.144706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d490e-65be-4b8b-9ad7-9af99c55bff6 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:07.161571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 838/1674] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:07.163004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d490e-65be-4b8b-9ad7-9af99c55bff6 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:07.173101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 837/1675] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:06 2022] POST /identity/v3/projects => generated 403 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:07.178022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4d095-aa03-4512-b260-33cf8b55c2be admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:07.185658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d490e-65be-4b8b-9ad7-9af99c55bff6 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:07.355740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733d9e9-80ab-46c7-ac2d-582dd9103322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:07.356112 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733d9e9-80ab-46c7-ac2d-582dd9103322 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:07.359674 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733d9e9-80ab-46c7-ac2d-582dd9103322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:07.360086 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733d9e9-80ab-46c7-ac2d-582dd9103322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:07.360923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b733d9e9-80ab-46c7-ac2d-582dd9103322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:07.362665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b733d9e9-80ab-46c7-ac2d-582dd9103322 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:07.364677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4d095-aa03-4512-b260-33cf8b55c2be admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:07.371235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a18dc-cc15-4a11-8e35-d96fe8c2101e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:07.372210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a18dc-cc15-4a11-8e35-d96fe8c2101e admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:07.373046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a18dc-cc15-4a11-8e35-d96fe8c2101e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:07.373525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a18dc-cc15-4a11-8e35-d96fe8c2101e admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:07.374533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a18dc-cc15-4a11-8e35-d96fe8c2101e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:07.378198 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a18dc-cc15-4a11-8e35-d96fe8c2101e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:07.393599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 839/1676] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:07.399450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733d9e9-80ab-46c7-ac2d-582dd9103322 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:07.418188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733d9e9-80ab-46c7-ac2d-582dd9103322 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:07.431895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0f031-dc89-453c-ac30-a8d10f912033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:07.434217 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0f031-dc89-453c-ac30-a8d10f912033 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:07.434650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0f031-dc89-453c-ac30-a8d10f912033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:07.435145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0f031-dc89-453c-ac30-a8d10f912033 admin admin] PATH_INFO: `/v3/projects/2720afb09ca44f5d999bad103f7dad5f/users/6deaa1346ec74804868dc269a1c8ea58/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:07.476703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0f031-dc89-453c-ac30-a8d10f912033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2720afb09ca44f5d999bad103f7dad5f, user_id=6deaa1346ec74804868dc269a1c8ea58, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:07.481355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0f031-dc89-453c-ac30-a8d10f912033 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:07.530960 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 838/1677] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:07 2022] POST /identity/v3/users => generated 302 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:07.533191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919f81c-544d-4a64-8766-81e01665885a None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:07.533842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919f81c-544d-4a64-8766-81e01665885a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:07.534535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919f81c-544d-4a64-8766-81e01665885a None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:07.543535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 839/1678] 10.222.0.85 () {54 vars in 912 bytes} [Mon Aug 29 20:27:07 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:07.550283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:07.552911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919f81c-544d-4a64-8766-81e01665885a None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:07.591782 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919f81c-544d-4a64-8766-81e01665885a None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:07.602173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5df5f-9f24-47e1-b555-ebe89ccca3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:07.602440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 840/1679] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:07 2022] PUT /identity/v3/projects/2720afb09ca44f5d999bad103f7dad5f/users/6deaa1346ec74804868dc269a1c8ea58/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:07.602910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5df5f-9f24-47e1-b555-ebe89ccca3fd admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:07.603451 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5df5f-9f24-47e1-b555-ebe89ccca3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:07.603584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314af71-af44-45a4-94f3-1aafc6d74b20 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:07.610700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314af71-af44-45a4-94f3-1aafc6d74b20 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:07.610700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314af71-af44-45a4-94f3-1aafc6d74b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:07.621249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5df5f-9f24-47e1-b555-ebe89ccca3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:07.622756 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5df5f-9f24-47e1-b555-ebe89ccca3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:07.625167 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5df5f-9f24-47e1-b555-ebe89ccca3fd admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:07.642530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 840/1680] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:07.653304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a15dc-7d41-44c7-b153-aa0186009f75 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:07.653669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a15dc-7d41-44c7-b153-aa0186009f75 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:07.653958 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a15dc-7d41-44c7-b153-aa0186009f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:07.690636 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-4314af71-af44-45a4-94f3-1aafc6d74b20 None None] MFA Rules not processed for user `aa222b1c89dc4d2cb1686b4838c7908b`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:07.693238 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-4314af71-af44-45a4-94f3-1aafc6d74b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:07.721045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-349a15dc-7d41-44c7-b153-aa0186009f75 None None] MFA Rules not processed for user `6deaa1346ec74804868dc269a1c8ea58`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:07.722409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-349a15dc-7d41-44c7-b153-aa0186009f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:07.772616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 841/1681] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:27:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:07.773483 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314af71-af44-45a4-94f3-1aafc6d74b20 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:07.847415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 841/1682] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:07.848526 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a15dc-7d41-44c7-b153-aa0186009f75 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:07.852593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-349a15dc-7d41-44c7-b153-aa0186009f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:07.867683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314af71-af44-45a4-94f3-1aafc6d74b20 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:07.871878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a15dc-7d41-44c7-b153-aa0186009f75 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:07.875923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2b91-d916-40d5-9d69-09d6661f5944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:07.876797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2b91-d916-40d5-9d69-09d6661f5944 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:07.877177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2b91-d916-40d5-9d69-09d6661f5944 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:07.877621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2b91-d916-40d5-9d69-09d6661f5944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:07.879542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a6c16-aa65-41eb-8e4d-d125e3589625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:07.880363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a6c16-aa65-41eb-8e4d-d125e3589625 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:07.880553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a6c16-aa65-41eb-8e4d-d125e3589625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:07.880824 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a6c16-aa65-41eb-8e4d-d125e3589625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:07.881488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a6c16-aa65-41eb-8e4d-d125e3589625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:07.883120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a6c16-aa65-41eb-8e4d-d125e3589625 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:07.908461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb2b91-d916-40d5-9d69-09d6661f5944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:07.912596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb2b91-d916-40d5-9d69-09d6661f5944 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:07.926571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 842/1683] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:07.927167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a6c16-aa65-41eb-8e4d-d125e3589625 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:07.942803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a6c16-aa65-41eb-8e4d-d125e3589625 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:07.952318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 842/1684] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:07.957535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4bcfa-1227-4ec9-9ff2-9ff9eae57432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:07.965193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4bcfa-1227-4ec9-9ff2-9ff9eae57432 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:07.965193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4bcfa-1227-4ec9-9ff2-9ff9eae57432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:07.965193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4bcfa-1227-4ec9-9ff2-9ff9eae57432 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:07.965193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4bcfa-1227-4ec9-9ff2-9ff9eae57432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:07.965585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2b91-d916-40d5-9d69-09d6661f5944 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:07.965665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4bcfa-1227-4ec9-9ff2-9ff9eae57432 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:07.989825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2b91-d916-40d5-9d69-09d6661f5944 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:07.996275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b33c3-8ea3-42a7-af4b-2f7b2ab2698c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:07.997475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b33c3-8ea3-42a7-af4b-2f7b2ab2698c admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:07.997939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b33c3-8ea3-42a7-af4b-2f7b2ab2698c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:07.998466 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b33c3-8ea3-42a7-af4b-2f7b2ab2698c admin admin] PATH_INFO: `/v3/projects/709e855dc6b0428d84a8f2cfeb7df063/users/8b0cd78e77ee4daab447eba771307ff6/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:08.039460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b33c3-8ea3-42a7-af4b-2f7b2ab2698c admin admin] RBAC: Authorizing `identity:create_grant(project_id=709e855dc6b0428d84a8f2cfeb7df063, user_id=8b0cd78e77ee4daab447eba771307ff6, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:08.050195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b33c3-8ea3-42a7-af4b-2f7b2ab2698c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:08.147673 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 843/1685] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:07 2022] POST /identity/v3/projects => generated 371 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:08.152240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4bcfa-1227-4ec9-9ff2-9ff9eae57432 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:08.203071 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 843/1686] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:07 2022] PUT /identity/v3/projects/709e855dc6b0428d84a8f2cfeb7df063/users/8b0cd78e77ee4daab447eba771307ff6/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:08.209866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d1881-699c-41ab-8293-be8754ac42f8 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:08.210781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d1881-699c-41ab-8293-be8754ac42f8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:08.211260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d1881-699c-41ab-8293-be8754ac42f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:08.268603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-c18d1881-699c-41ab-8293-be8754ac42f8 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:08.270066 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c18d1881-699c-41ab-8293-be8754ac42f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:08.383073 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4bcfa-1227-4ec9-9ff2-9ff9eae57432 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:08.483141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 844/1687] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:08.509670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:08.552724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5473f8-abba-4788-a48c-c759cd715e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:08.553720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5473f8-abba-4788-a48c-c759cd715e84 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:08.554370 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5473f8-abba-4788-a48c-c759cd715e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:08.554843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5473f8-abba-4788-a48c-c759cd715e84 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:08.555681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5473f8-abba-4788-a48c-c759cd715e84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:08.557034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d1881-699c-41ab-8293-be8754ac42f8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:08.559398 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5473f8-abba-4788-a48c-c759cd715e84 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:08.560254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c18d1881-699c-41ab-8293-be8754ac42f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:08.653947 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 844/1688] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:08 2022] POST /identity/v3/users => generated 292 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:08.657424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5473f8-abba-4788-a48c-c759cd715e84 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:08.686030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5473f8-abba-4788-a48c-c759cd715e84 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:08.709058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c931d5-54fa-44d6-abcc-25917bdfe929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:08.709929 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c931d5-54fa-44d6-abcc-25917bdfe929 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:08.710353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c931d5-54fa-44d6-abcc-25917bdfe929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:08.712037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c931d5-54fa-44d6-abcc-25917bdfe929 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:08.715576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c931d5-54fa-44d6-abcc-25917bdfe929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:08.715576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c931d5-54fa-44d6-abcc-25917bdfe929 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:08.721564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d1881-699c-41ab-8293-be8754ac42f8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:08.738381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f1a90-99c2-4b5c-92ef-a52df07424c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:08.739025 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f1a90-99c2-4b5c-92ef-a52df07424c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:08.739311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f1a90-99c2-4b5c-92ef-a52df07424c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:08.739585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f1a90-99c2-4b5c-92ef-a52df07424c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:08.744700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 845/1689] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:08.758144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c931d5-54fa-44d6-abcc-25917bdfe929 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:08.761568 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f1a90-99c2-4b5c-92ef-a52df07424c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:08.763423 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f1a90-99c2-4b5c-92ef-a52df07424c2 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:08.774731 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:08.786656 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 845/1690] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:27:08.791056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c931d5-54fa-44d6-abcc-25917bdfe929 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:08.813495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07da33-97da-4e79-b37f-ad6679c0d885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:08.814193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07da33-97da-4e79-b37f-ad6679c0d885 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:08.814477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07da33-97da-4e79-b37f-ad6679c0d885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:08.814959 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07da33-97da-4e79-b37f-ad6679c0d885 admin admin] PATH_INFO: `/v3/projects/51412cdf82ff403489813c2e8177ed1f/users/8410712ccf0f47068578394c2a817e31/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:08.889887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07da33-97da-4e79-b37f-ad6679c0d885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51412cdf82ff403489813c2e8177ed1f, user_id=8410712ccf0f47068578394c2a817e31, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:08.893459 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07da33-97da-4e79-b37f-ad6679c0d885 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:08.933028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:08.988485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 846/1691] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:08 2022] PUT /identity/v3/projects/51412cdf82ff403489813c2e8177ed1f/users/8410712ccf0f47068578394c2a817e31/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:08.988748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f1a90-99c2-4b5c-92ef-a52df07424c2 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:09.001509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f1a90-99c2-4b5c-92ef-a52df07424c2 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:09.125171 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede493e-37ae-457c-9887-2182c4c1a64e None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:09.125607 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede493e-37ae-457c-9887-2182c4c1a64e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:09.126067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede493e-37ae-457c-9887-2182c4c1a64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:09.166970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d09a51-a252-437c-8e21-07bccbef6a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:09.169756 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d09a51-a252-437c-8e21-07bccbef6a98 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:09.170266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d09a51-a252-437c-8e21-07bccbef6a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:09.170881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d09a51-a252-437c-8e21-07bccbef6a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:09.173050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d09a51-a252-437c-8e21-07bccbef6a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:09.195572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d09a51-a252-437c-8e21-07bccbef6a98 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:09.213136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 846/1692] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:09.214064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d09a51-a252-437c-8e21-07bccbef6a98 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:09.221231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-9ede493e-37ae-457c-9887-2182c4c1a64e None None] MFA Rules not processed for user `336345e5437341ba9859d776c44ea4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:09.223258 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9ede493e-37ae-457c-9887-2182c4c1a64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:09.235759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d09a51-a252-437c-8e21-07bccbef6a98 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:09.244830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307472e9-8cac-483a-8f75-4f8612efb703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:09.246132 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307472e9-8cac-483a-8f75-4f8612efb703 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:09.246488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307472e9-8cac-483a-8f75-4f8612efb703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:09.246764 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307472e9-8cac-483a-8f75-4f8612efb703 admin admin] PATH_INFO: `/v3/projects/51412cdf82ff403489813c2e8177ed1f/users/8410712ccf0f47068578394c2a817e31/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:09.249125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307472e9-8cac-483a-8f75-4f8612efb703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51412cdf82ff403489813c2e8177ed1f, user_id=8410712ccf0f47068578394c2a817e31, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:09.252474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307472e9-8cac-483a-8f75-4f8612efb703 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:09.295617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 847/1693] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:09 2022] PUT /identity/v3/projects/51412cdf82ff403489813c2e8177ed1f/users/8410712ccf0f47068578394c2a817e31/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:09.297881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbbef1-82ff-47b1-8de2-c1ef04f2554d None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:09.300499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbbef1-82ff-47b1-8de2-c1ef04f2554d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:09.300944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbbef1-82ff-47b1-8de2-c1ef04f2554d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:09.389770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 847/1694] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:09.400522 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-30dbbef1-82ff-47b1-8de2-c1ef04f2554d None None] MFA Rules not processed for user `8410712ccf0f47068578394c2a817e31`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:09.402254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-30dbbef1-82ff-47b1-8de2-c1ef04f2554d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:09.403142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a8530-021a-41e9-8135-4be27e1b3823 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:09.403674 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a8530-021a-41e9-8135-4be27e1b3823 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:09.405026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a8530-021a-41e9-8135-4be27e1b3823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:09.498495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-7c5a8530-021a-41e9-8135-4be27e1b3823 None None] MFA Rules not processed for user `336345e5437341ba9859d776c44ea4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:09.502501 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7c5a8530-021a-41e9-8135-4be27e1b3823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:09.658889 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:09.688919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 848/1695] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:09.690764 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbbef1-82ff-47b1-8de2-c1ef04f2554d None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:09.695320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-30dbbef1-82ff-47b1-8de2-c1ef04f2554d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:09.700269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 848/1696] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:09.701135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a8530-021a-41e9-8135-4be27e1b3823 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:09.708576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7c5a8530-021a-41e9-8135-4be27e1b3823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:09.863066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a8530-021a-41e9-8135-4be27e1b3823 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:09.873484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbbef1-82ff-47b1-8de2-c1ef04f2554d None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:09.874762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b3664-3520-4003-b48a-cdddec2a735c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:09.875397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b3664-3520-4003-b48a-cdddec2a735c service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:09.875679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b3664-3520-4003-b48a-cdddec2a735c service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:09.875957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b3664-3520-4003-b48a-cdddec2a735c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:09.894903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b84ea-dd67-425f-bb25-360b549ee925 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:09.902715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b84ea-dd67-425f-bb25-360b549ee925 service placement] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:09.902715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b84ea-dd67-425f-bb25-360b549ee925 service placement] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:09.902715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b84ea-dd67-425f-bb25-360b549ee925 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:09.902715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8e0b84ea-dd67-425f-bb25-360b549ee925 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:09.913197 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b3664-3520-4003-b48a-cdddec2a735c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:09.915246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b3664-3520-4003-b48a-cdddec2a735c service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:09.942550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b84ea-dd67-425f-bb25-360b549ee925 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:09.944830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b84ea-dd67-425f-bb25-360b549ee925 service placement] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:09.946076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 849/1697] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:09.947254 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b3664-3520-4003-b48a-cdddec2a735c service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:09.948860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-448b3664-3520-4003-b48a-cdddec2a735c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:09.984587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 849/1698] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:10.129506 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b3664-3520-4003-b48a-cdddec2a735c service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:10.151015 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71213ce6-f86c-4763-b0f0-3ccd71fa8b76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:10.153040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71213ce6-f86c-4763-b0f0-3ccd71fa8b76 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:10.155357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71213ce6-f86c-4763-b0f0-3ccd71fa8b76 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:10.155664 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71213ce6-f86c-4763-b0f0-3ccd71fa8b76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:10.178523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71213ce6-f86c-4763-b0f0-3ccd71fa8b76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:10.182016 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71213ce6-f86c-4763-b0f0-3ccd71fa8b76 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:10.200639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 850/1699] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:09 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:10.353525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b546-e849-4980-866c-63e9e0db4e6b None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:10.353981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b546-e849-4980-866c-63e9e0db4e6b None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:10.355520 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b546-e849-4980-866c-63e9e0db4e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:10.443003 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-e596b546-e849-4980-866c-63e9e0db4e6b None None] MFA Rules not processed for user `3f4b05a527e14d2599e03077c6c5f96f`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:10.445103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e596b546-e849-4980-866c-63e9e0db4e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:10.498663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:10.594845 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 850/1700] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:10.614930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71213ce6-f86c-4763-b0f0-3ccd71fa8b76 service nova] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:10.618528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-71213ce6-f86c-4763-b0f0-3ccd71fa8b76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:10.650840 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71213ce6-f86c-4763-b0f0-3ccd71fa8b76 service nova] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:10.667231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33a6ed-5d61-47b4-a501-fbf13b20ae5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:10.669109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33a6ed-5d61-47b4-a501-fbf13b20ae5a service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:10.669419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33a6ed-5d61-47b4-a501-fbf13b20ae5a service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:10.669704 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33a6ed-5d61-47b4-a501-fbf13b20ae5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:10.680674 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33a6ed-5d61-47b4-a501-fbf13b20ae5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:10.682332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33a6ed-5d61-47b4-a501-fbf13b20ae5a service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:10.713341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 851/1701] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:11.107196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:11.421763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b546-e849-4980-866c-63e9e0db4e6b None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:11.617466 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b546-e849-4980-866c-63e9e0db4e6b None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:11.619142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4791b-c3e9-47a1-820e-f7001cf49f8d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:11.619670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4791b-c3e9-47a1-820e-f7001cf49f8d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:11.619740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4791b-c3e9-47a1-820e-f7001cf49f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:11.633239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:11.633902 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:11.634341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:11.634765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] PATH_INFO: `/v3/users/6deaa1346ec74804868dc269a1c8ea58` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:11.636312 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6deaa1346ec74804868dc269a1c8ea58)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:11.638425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:11.696040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-14d4791b-c3e9-47a1-820e-f7001cf49f8d None None] MFA Rules not processed for user `8b0cd78e77ee4daab447eba771307ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:11.699492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-14d4791b-c3e9-47a1-820e-f7001cf49f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:11.891208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 852/1702] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:11 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:11.891208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4791b-c3e9-47a1-820e-f7001cf49f8d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:11.934527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4791b-c3e9-47a1-820e-f7001cf49f8d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:11.948311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610820e8-41bb-4d49-8b8e-635e0b4dafb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:11.949488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610820e8-41bb-4d49-8b8e-635e0b4dafb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:11.949772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610820e8-41bb-4d49-8b8e-635e0b4dafb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:11.950047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610820e8-41bb-4d49-8b8e-635e0b4dafb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:11.953240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610820e8-41bb-4d49-8b8e-635e0b4dafb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:11.965513 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610820e8-41bb-4d49-8b8e-635e0b4dafb4 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:11.987743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6deaa1346ec74804868dc269a1c8ea58'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:12.000081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6deaa1346ec74804868dc269a1c8ea58'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:12.006881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6deaa1346ec74804868dc269a1c8ea58'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:12.222815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6deaa1346ec74804868dc269a1c8ea58'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:12.225921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 853/1703] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:11 2022] POST /identity/v3/projects => generated 403 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:12.227387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7917eb-c33d-486a-83c7-65e178dbc402 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:12.227906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7917eb-c33d-486a-83c7-65e178dbc402 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:12.228336 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7917eb-c33d-486a-83c7-65e178dbc402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:12.229357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6deaa1346ec74804868dc269a1c8ea58'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:12.274215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3a7917eb-c33d-486a-83c7-65e178dbc402 None None] MFA Rules not processed for user `336345e5437341ba9859d776c44ea4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:12.274576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3a7917eb-c33d-486a-83c7-65e178dbc402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:12.317874 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 851/1704] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:11 2022] DELETE /identity/v3/users/6deaa1346ec74804868dc269a1c8ea58 => generated 0 bytes in 898 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:12.321885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:12.325136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:12.466077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 854/1705] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:12.481785 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3682fe0-0ba5-4d99-b731-5c84e3573fc2 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:12.482982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3682fe0-0ba5-4d99-b731-5c84e3573fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:12.487128 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3682fe0-0ba5-4d99-b731-5c84e3573fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:12.537115 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067412ea-faef-4d55-b67d-c396d1bb8782 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:12.569879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e3682fe0-0ba5-4d99-b731-5c84e3573fc2 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:12.570954 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e3682fe0-0ba5-4d99-b731-5c84e3573fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:12.587664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e0c91-2c00-4a5b-afa0-64a74f83da5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:12.589188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e0c91-2c00-4a5b-afa0-64a74f83da5b admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:12.593095 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e0c91-2c00-4a5b-afa0-64a74f83da5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:12.593890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e0c91-2c00-4a5b-afa0-64a74f83da5b admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:12.595184 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e0c91-2c00-4a5b-afa0-64a74f83da5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:12.600588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e0c91-2c00-4a5b-afa0-64a74f83da5b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:12.606416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:12.618131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 855/1706] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:12.619130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3682fe0-0ba5-4d99-b731-5c84e3573fc2 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:12.625825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e3682fe0-0ba5-4d99-b731-5c84e3573fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:12.739623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 852/1707] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:12 2022] POST /identity/v3/users => generated 308 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:12.747070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e0c91-2c00-4a5b-afa0-64a74f83da5b admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:12.770060 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3682fe0-0ba5-4d99-b731-5c84e3573fc2 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:12.782864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d49e0-db63-4b20-bee5-7f3068772fd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:12.783752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d49e0-db63-4b20-bee5-7f3068772fd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:12.784269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d49e0-db63-4b20-bee5-7f3068772fd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:12.784897 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d49e0-db63-4b20-bee5-7f3068772fd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:12.800940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e0c91-2c00-4a5b-afa0-64a74f83da5b admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:12.812350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d6bac-8da9-4a54-af08-859cab8c37f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:12.812983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d6bac-8da9-4a54-af08-859cab8c37f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:12.815071 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d6bac-8da9-4a54-af08-859cab8c37f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:12.816934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d6bac-8da9-4a54-af08-859cab8c37f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:12.817873 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d6bac-8da9-4a54-af08-859cab8c37f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:12.821845 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d49e0-db63-4b20-bee5-7f3068772fd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:12.828402 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d6bac-8da9-4a54-af08-859cab8c37f9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:12.829326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d49e0-db63-4b20-bee5-7f3068772fd5 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:12.842599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 856/1708] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:12.868338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 853/1709] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:12.872786 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d49e0-db63-4b20-bee5-7f3068772fd5 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:12.889102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d49e0-db63-4b20-bee5-7f3068772fd5 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:12.907116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3595da8-1824-44f0-bd54-bf34f85091b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:12.907962 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3595da8-1824-44f0-bd54-bf34f85091b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:12.909451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3595da8-1824-44f0-bd54-bf34f85091b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:12.909937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3595da8-1824-44f0-bd54-bf34f85091b1 admin admin] PATH_INFO: `/v3/projects/8ce8eadad3854460a539325e16b35d26/users/29d2da8e56cf4360bf0a04bbf055eebd/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:12.965299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3595da8-1824-44f0-bd54-bf34f85091b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ce8eadad3854460a539325e16b35d26, user_id=29d2da8e56cf4360bf0a04bbf055eebd, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:12.968721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3595da8-1824-44f0-bd54-bf34f85091b1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:13.098517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 857/1710] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:12 2022] PUT /identity/v3/projects/8ce8eadad3854460a539325e16b35d26/users/29d2da8e56cf4360bf0a04bbf055eebd/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:13.102332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d6bac-8da9-4a54-af08-859cab8c37f9 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:13.125967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d6bac-8da9-4a54-af08-859cab8c37f9 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:13.189589 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:13.229355 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84e643-5470-4224-b3c7-c9da40c55d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:13.230824 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84e643-5470-4224-b3c7-c9da40c55d8b admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:13.231468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84e643-5470-4224-b3c7-c9da40c55d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:13.232475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84e643-5470-4224-b3c7-c9da40c55d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:13.233195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84e643-5470-4224-b3c7-c9da40c55d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:13.235192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84e643-5470-4224-b3c7-c9da40c55d8b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:13.250323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 854/1711] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:13.254378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3595da8-1824-44f0-bd54-bf34f85091b1 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:13.269996 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3595da8-1824-44f0-bd54-bf34f85091b1 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:13.279718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dbb77-5189-491e-aadb-c91c6a67eb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:13.281373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dbb77-5189-491e-aadb-c91c6a67eb13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:13.281833 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dbb77-5189-491e-aadb-c91c6a67eb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:13.282301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dbb77-5189-491e-aadb-c91c6a67eb13 admin admin] PATH_INFO: `/v3/projects/8ce8eadad3854460a539325e16b35d26/users/29d2da8e56cf4360bf0a04bbf055eebd/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:13.288973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780dbb77-5189-491e-aadb-c91c6a67eb13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ce8eadad3854460a539325e16b35d26, user_id=29d2da8e56cf4360bf0a04bbf055eebd, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:13.297200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780dbb77-5189-491e-aadb-c91c6a67eb13 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:13.335541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 858/1712] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:13 2022] PUT /identity/v3/projects/8ce8eadad3854460a539325e16b35d26/users/29d2da8e56cf4360bf0a04bbf055eebd/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:14.361545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:14.931583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84e643-5470-4224-b3c7-c9da40c55d8b admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:14.934041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ae84e643-5470-4224-b3c7-c9da40c55d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:14.954845 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84e643-5470-4224-b3c7-c9da40c55d8b admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:15.156739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:15.159375 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:15.160338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:15.160338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] PATH_INFO: `/v3/projects/2720afb09ca44f5d999bad103f7dad5f` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:15.161624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2720afb09ca44f5d999bad103f7dad5f)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:15.163876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:15.305269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:15.314862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2720afb09ca44f5d999bad103f7dad5f', 'request_id': 'req-981a8f5e-0331-4f9e-b522-d6cd2fa32226'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:15.328396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2720afb09ca44f5d999bad103f7dad5f', 'request_id': 'req-981a8f5e-0331-4f9e-b522-d6cd2fa32226'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:15.457918 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2720afb09ca44f5d999bad103f7dad5f', 'request_id': 'req-981a8f5e-0331-4f9e-b522-d6cd2fa32226'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:15.489313 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] The token cache is being invalidate because project 2720afb09ca44f5d999bad103f7dad5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:15.489625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:15.502122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:15.502122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 855/1713] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:14 2022] DELETE /identity/v3/projects/2720afb09ca44f5d999bad103f7dad5f => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:15.514883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dbb77-5189-491e-aadb-c91c6a67eb13 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:15.515553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:15.515981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:15.517640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-780dbb77-5189-491e-aadb-c91c6a67eb13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:15.541850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dbb77-5189-491e-aadb-c91c6a67eb13 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:15.692126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a8f5e-0331-4f9e-b522-d6cd2fa32226 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:15.710568 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4b46a-787f-4fec-a4c1-61453277d3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:15.711394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4b46a-787f-4fec-a4c1-61453277d3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:15.711751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4b46a-787f-4fec-a4c1-61453277d3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:15.712140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4b46a-787f-4fec-a4c1-61453277d3d1 admin admin] PATH_INFO: `/v3/users/2311207200ea41169272e4097aa85c38` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:15.713532 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4b46a-787f-4fec-a4c1-61453277d3d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2311207200ea41169272e4097aa85c38)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:15.716739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4b46a-787f-4fec-a4c1-61453277d3d1 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:15.727087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d7066-a76e-4152-80ed-66e4c6411037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:15.728533 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d7066-a76e-4152-80ed-66e4c6411037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:15.729408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d7066-a76e-4152-80ed-66e4c6411037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:15.737487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d7066-a76e-4152-80ed-66e4c6411037 admin admin] PATH_INFO: `/v3/users/8410712ccf0f47068578394c2a817e31` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:15.744932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d7066-a76e-4152-80ed-66e4c6411037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8410712ccf0f47068578394c2a817e31)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:15.752581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d7066-a76e-4152-80ed-66e4c6411037 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:16.152035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e00d7066-a76e-4152-80ed-66e4c6411037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8410712ccf0f47068578394c2a817e31'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:16.160373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e00d7066-a76e-4152-80ed-66e4c6411037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8410712ccf0f47068578394c2a817e31'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:16.170620 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:16.175063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-69d4b46a-787f-4fec-a4c1-61453277d3d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2311207200ea41169272e4097aa85c38'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:16.175813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-69d4b46a-787f-4fec-a4c1-61453277d3d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2311207200ea41169272e4097aa85c38'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:16.188899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-69d4b46a-787f-4fec-a4c1-61453277d3d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2311207200ea41169272e4097aa85c38'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:16.258023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e00d7066-a76e-4152-80ed-66e4c6411037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8410712ccf0f47068578394c2a817e31'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:16.258835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e00d7066-a76e-4152-80ed-66e4c6411037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8410712ccf0f47068578394c2a817e31'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:16.290076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e00d7066-a76e-4152-80ed-66e4c6411037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8410712ccf0f47068578394c2a817e31'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:16.311229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 859/1714] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:15 2022] DELETE /identity/v3/users/8410712ccf0f47068578394c2a817e31 => generated 0 bytes in 798 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:16.337669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-69d4b46a-787f-4fec-a4c1-61453277d3d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2311207200ea41169272e4097aa85c38'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:16.338754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-69d4b46a-787f-4fec-a4c1-61453277d3d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2311207200ea41169272e4097aa85c38'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:16.366383 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 856/1715] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:15 2022] DELETE /identity/v3/users/2311207200ea41169272e4097aa85c38 => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:16.375885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0a067-bb2b-4df1-843d-86320b4efde0 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:16.376431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0a067-bb2b-4df1-843d-86320b4efde0 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:16.376871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0a067-bb2b-4df1-843d-86320b4efde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:16.411892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-72f0a067-bb2b-4df1-843d-86320b4efde0 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:16.413395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-72f0a067-bb2b-4df1-843d-86320b4efde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:16.502364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c518b2a-0db2-4e90-9f8d-b07cee350a33 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:16.502588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c518b2a-0db2-4e90-9f8d-b07cee350a33 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:16.502882 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c518b2a-0db2-4e90-9f8d-b07cee350a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:16.608228 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7c518b2a-0db2-4e90-9f8d-b07cee350a33 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:16.609763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7c518b2a-0db2-4e90-9f8d-b07cee350a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:16.614822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 860/1716] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:16.616396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8921647-c5b3-47f7-8d39-75bb48f3803f None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:16.616955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8921647-c5b3-47f7-8d39-75bb48f3803f None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:16.620606 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8921647-c5b3-47f7-8d39-75bb48f3803f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:16.674645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e8921647-c5b3-47f7-8d39-75bb48f3803f None None] MFA Rules not processed for user `3f4b05a527e14d2599e03077c6c5f96f`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:16.676073 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e8921647-c5b3-47f7-8d39-75bb48f3803f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:16.761603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 857/1717] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:27:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:16.767388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c518b2a-0db2-4e90-9f8d-b07cee350a33 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:16.767388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7c518b2a-0db2-4e90-9f8d-b07cee350a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:16.799474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 861/1718] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:16.888328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c518b2a-0db2-4e90-9f8d-b07cee350a33 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:16.893969 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd5f55-60d5-48d7-a6e3-4c315f9df5b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:16.897710 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd5f55-60d5-48d7-a6e3-4c315f9df5b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:16.898153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd5f55-60d5-48d7-a6e3-4c315f9df5b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:16.898570 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd5f55-60d5-48d7-a6e3-4c315f9df5b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:16.945048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd5f55-60d5-48d7-a6e3-4c315f9df5b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:16.952509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd5f55-60d5-48d7-a6e3-4c315f9df5b3 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:16.965086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 858/1719] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:17.177553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8921647-c5b3-47f7-8d39-75bb48f3803f None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:17.179209 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e8921647-c5b3-47f7-8d39-75bb48f3803f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:17.370045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd5f55-60d5-48d7-a6e3-4c315f9df5b3 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:17.372884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-10bd5f55-60d5-48d7-a6e3-4c315f9df5b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:17.407196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd5f55-60d5-48d7-a6e3-4c315f9df5b3 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:17.434837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:17.438143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:17.438489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:17.438789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] PATH_INFO: `/v3/projects/51412cdf82ff403489813c2e8177ed1f` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:17.451742 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51412cdf82ff403489813c2e8177ed1f)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:17.462103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:17.486056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8921647-c5b3-47f7-8d39-75bb48f3803f None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:17.517353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4faa28-7e21-4692-a80a-f286f3aefb82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:17.522235 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4faa28-7e21-4692-a80a-f286f3aefb82 service swift] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:17.524188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4faa28-7e21-4692-a80a-f286f3aefb82 service swift] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:17.524759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4faa28-7e21-4692-a80a-f286f3aefb82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:17.552897 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4faa28-7e21-4692-a80a-f286f3aefb82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:17.600896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:17.602349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4faa28-7e21-4692-a80a-f286f3aefb82 service swift] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:17.656556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 862/1720] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:27:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:27:17.680901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51412cdf82ff403489813c2e8177ed1f', 'request_id': 'req-ad885e09-31e6-4991-914a-28dbd7960085'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:17.695869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51412cdf82ff403489813c2e8177ed1f', 'request_id': 'req-ad885e09-31e6-4991-914a-28dbd7960085'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:17.806215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51412cdf82ff403489813c2e8177ed1f', 'request_id': 'req-ad885e09-31e6-4991-914a-28dbd7960085'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:17.837867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] The token cache is being invalidate because project 51412cdf82ff403489813c2e8177ed1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:17.838283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:17.840570 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:17.843702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 859/1721] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:17 2022] DELETE /identity/v3/projects/51412cdf82ff403489813c2e8177ed1f => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:17.851250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2f06f-bad0-450e-b021-322e3cd9788d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:17.851472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2f06f-bad0-450e-b021-322e3cd9788d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:17.851738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2f06f-bad0-450e-b021-322e3cd9788d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:17.900146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d3c2f06f-bad0-450e-b021-322e3cd9788d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:17.901475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d3c2f06f-bad0-450e-b021-322e3cd9788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:18.135132 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:18.137766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:18.148700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 863/1722] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:18.157604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3445630-1551-47ed-a143-0bf8a123e2fe None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:18.157604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3445630-1551-47ed-a143-0bf8a123e2fe None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:18.157604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3445630-1551-47ed-a143-0bf8a123e2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:18.169604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad885e09-31e6-4991-914a-28dbd7960085 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:18.193673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d66d28-d340-4213-a501-3483d170525c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:18.194357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d66d28-d340-4213-a501-3483d170525c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:18.194641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d66d28-d340-4213-a501-3483d170525c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:18.194913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d66d28-d340-4213-a501-3483d170525c admin admin] PATH_INFO: `/v3/projects/fab91f53f7d0452189b5977d835ffb24` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:18.213144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d66d28-d340-4213-a501-3483d170525c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab91f53f7d0452189b5977d835ffb24)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:18.215147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d66d28-d340-4213-a501-3483d170525c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:18.240850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-f3445630-1551-47ed-a143-0bf8a123e2fe None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:18.242103 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-f3445630-1551-47ed-a143-0bf8a123e2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:18.260917 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 864/1723] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:18.266611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3445630-1551-47ed-a143-0bf8a123e2fe None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:18.290238 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3445630-1551-47ed-a143-0bf8a123e2fe None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:18.303082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e711df4-a958-4b2f-82f5-f13c2957a4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:18.304340 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e711df4-a958-4b2f-82f5-f13c2957a4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:18.304771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e711df4-a958-4b2f-82f5-f13c2957a4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:18.311376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e711df4-a958-4b2f-82f5-f13c2957a4e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:18.311376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e711df4-a958-4b2f-82f5-f13c2957a4e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:18.311376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e711df4-a958-4b2f-82f5-f13c2957a4e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:18.323085 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e711df4-a958-4b2f-82f5-f13c2957a4e4 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:18.338244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 865/1724] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:18.343175 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e711df4-a958-4b2f-82f5-f13c2957a4e4 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:18.356027 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e711df4-a958-4b2f-82f5-f13c2957a4e4 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:18.361970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed7289-98a8-45f5-af6c-b99371e4d6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:18.362803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed7289-98a8-45f5-af6c-b99371e4d6cf admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:18.363233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed7289-98a8-45f5-af6c-b99371e4d6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:18.363689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed7289-98a8-45f5-af6c-b99371e4d6cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:18.364714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed7289-98a8-45f5-af6c-b99371e4d6cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:18.366795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed7289-98a8-45f5-af6c-b99371e4d6cf admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:18.424587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e0d66d28-d340-4213-a501-3483d170525c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab91f53f7d0452189b5977d835ffb24', 'request_id': 'req-e0d66d28-d340-4213-a501-3483d170525c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:18.431888 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e0d66d28-d340-4213-a501-3483d170525c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab91f53f7d0452189b5977d835ffb24', 'request_id': 'req-e0d66d28-d340-4213-a501-3483d170525c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:18.640743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e0d66d28-d340-4213-a501-3483d170525c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab91f53f7d0452189b5977d835ffb24', 'request_id': 'req-e0d66d28-d340-4213-a501-3483d170525c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:18.665790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e0d66d28-d340-4213-a501-3483d170525c admin admin] The token cache is being invalidate because project fab91f53f7d0452189b5977d835ffb24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:18.665940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 866/1725] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:18 2022] POST /identity/v3/projects => generated 369 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:27:18.666416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e0d66d28-d340-4213-a501-3483d170525c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:18.666416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e0d66d28-d340-4213-a501-3483d170525c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:18.666416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 860/1726] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:18 2022] DELETE /identity/v3/projects/fab91f53f7d0452189b5977d835ffb24 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:18.666669 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed7289-98a8-45f5-af6c-b99371e4d6cf admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:18.668904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba14ada-0096-499d-88c2-6efe9bfae605 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:18.669428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba14ada-0096-499d-88c2-6efe9bfae605 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:18.669975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-55ed7289-98a8-45f5-af6c-b99371e4d6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:18.670196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba14ada-0096-499d-88c2-6efe9bfae605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:18.737206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-8ba14ada-0096-499d-88c2-6efe9bfae605 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:18.738642 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8ba14ada-0096-499d-88c2-6efe9bfae605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:18.798167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed7289-98a8-45f5-af6c-b99371e4d6cf admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:18.803032 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3f52f-c76d-4936-b364-6e2c2caf80b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:18.803737 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3f52f-c76d-4936-b364-6e2c2caf80b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:18.804092 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3f52f-c76d-4936-b364-6e2c2caf80b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:18.804489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3f52f-c76d-4936-b364-6e2c2caf80b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:18.805204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3f52f-c76d-4936-b364-6e2c2caf80b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:18.807358 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3f52f-c76d-4936-b364-6e2c2caf80b5 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:18.868461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 861/1727] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:18.870067 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cc98e-7ad6-4a88-be72-fd93b0b5c188 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:18.870398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cc98e-7ad6-4a88-be72-fd93b0b5c188 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:18.870683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cc98e-7ad6-4a88-be72-fd93b0b5c188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:18.913447 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-2b0cc98e-7ad6-4a88-be72-fd93b0b5c188 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:18.923213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2b0cc98e-7ad6-4a88-be72-fd93b0b5c188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:18.941639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 862/1728] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:18.944803 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cc98e-7ad6-4a88-be72-fd93b0b5c188 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:18.958437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cc98e-7ad6-4a88-be72-fd93b0b5c188 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:18.966401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3786400-d98c-40c0-88ba-1a957764d75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:18.967087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3786400-d98c-40c0-88ba-1a957764d75d admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:18.967377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3786400-d98c-40c0-88ba-1a957764d75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:18.967649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3786400-d98c-40c0-88ba-1a957764d75d admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:18.968618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3786400-d98c-40c0-88ba-1a957764d75d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:18.968980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3786400-d98c-40c0-88ba-1a957764d75d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:18.970950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3786400-d98c-40c0-88ba-1a957764d75d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:18.992944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 863/1729] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:18.997145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3786400-d98c-40c0-88ba-1a957764d75d admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:19.009459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3786400-d98c-40c0-88ba-1a957764d75d admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:19.015736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7254d-d822-4ef2-b648-3b501e5b65cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:19.016507 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7254d-d822-4ef2-b648-3b501e5b65cc admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:19.016760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7254d-d822-4ef2-b648-3b501e5b65cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:19.017292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7254d-d822-4ef2-b648-3b501e5b65cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:19.018024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7254d-d822-4ef2-b648-3b501e5b65cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:19.019896 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7254d-d822-4ef2-b648-3b501e5b65cc admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:20.479649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 867/1730] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:18 2022] POST /identity/v3/users => generated 285 bytes in 1815 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:20.483690 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3f52f-c76d-4936-b364-6e2c2caf80b5 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:20.484913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 864/1731] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:18 2022] POST /identity/v3/projects => generated 381 bytes in 1488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:20.487984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7254d-d822-4ef2-b648-3b501e5b65cc admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:20.519292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3f52f-c76d-4936-b364-6e2c2caf80b5 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:20.666817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c3bcd-e062-4034-bde7-3a28f8b31bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:20.667797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c3bcd-e062-4034-bde7-3a28f8b31bbb admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:20.668959 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c3bcd-e062-4034-bde7-3a28f8b31bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:20.669311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c3bcd-e062-4034-bde7-3a28f8b31bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:20.670021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c3bcd-e062-4034-bde7-3a28f8b31bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:20.675249 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c3bcd-e062-4034-bde7-3a28f8b31bbb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:20.710248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 868/1732] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:20.712022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c3bcd-e062-4034-bde7-3a28f8b31bbb admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:20.742960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c3bcd-e062-4034-bde7-3a28f8b31bbb admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:20.767508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486860e7-1ef7-45ae-9886-bb18b04ae69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:20.770980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486860e7-1ef7-45ae-9886-bb18b04ae69b admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:20.771824 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486860e7-1ef7-45ae-9886-bb18b04ae69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:20.774444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486860e7-1ef7-45ae-9886-bb18b04ae69b admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:20.777886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486860e7-1ef7-45ae-9886-bb18b04ae69b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:20.782222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486860e7-1ef7-45ae-9886-bb18b04ae69b admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:20.829194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7254d-d822-4ef2-b648-3b501e5b65cc admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:20.843290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 869/1733] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:20.855653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486860e7-1ef7-45ae-9886-bb18b04ae69b admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:20.858997 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a615d-efde-4878-908a-47cf7168fd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:20.861362 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a615d-efde-4878-908a-47cf7168fd2c admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:20.862910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a615d-efde-4878-908a-47cf7168fd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:20.863941 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a615d-efde-4878-908a-47cf7168fd2c admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:20.866168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a615d-efde-4878-908a-47cf7168fd2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:20.875408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a615d-efde-4878-908a-47cf7168fd2c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:20.907074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486860e7-1ef7-45ae-9886-bb18b04ae69b admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:20.926427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba6574-b4ea-49f4-83eb-9b2e8ddf0f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:20.927213 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba6574-b4ea-49f4-83eb-9b2e8ddf0f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:20.929898 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba6574-b4ea-49f4-83eb-9b2e8ddf0f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:20.931062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba6574-b4ea-49f4-83eb-9b2e8ddf0f11 admin admin] PATH_INFO: `/v3/projects/b11ae826af3546658931245e764d270b/users/dae0a0e651c745a9a9f42e1485832bc9/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:20.960362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba6574-b4ea-49f4-83eb-9b2e8ddf0f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b11ae826af3546658931245e764d270b, user_id=dae0a0e651c745a9a9f42e1485832bc9, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:20.960362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba6574-b4ea-49f4-83eb-9b2e8ddf0f11 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:21.029113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:21.041736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 865/1734] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:20 2022] POST /identity/v3/users => generated 291 bytes in 553 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:21.041736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a615d-efde-4878-908a-47cf7168fd2c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:21.067010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a615d-efde-4878-908a-47cf7168fd2c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:21.080624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c95e2e-27c4-4e80-a7ce-87a4142675df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:21.081199 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c95e2e-27c4-4e80-a7ce-87a4142675df admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:21.081629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c95e2e-27c4-4e80-a7ce-87a4142675df admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:21.081919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c95e2e-27c4-4e80-a7ce-87a4142675df admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:21.082585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c95e2e-27c4-4e80-a7ce-87a4142675df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:21.089310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c95e2e-27c4-4e80-a7ce-87a4142675df admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:21.119272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 866/1735] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:21.121121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c88b40-aa90-4c7f-8551-46a2a0a8f5e1 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:21.121417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c88b40-aa90-4c7f-8551-46a2a0a8f5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:21.121698 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c88b40-aa90-4c7f-8551-46a2a0a8f5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:21.131790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 870/1736] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:20 2022] PUT /identity/v3/projects/b11ae826af3546658931245e764d270b/users/dae0a0e651c745a9a9f42e1485832bc9/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:21.134585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba6574-b4ea-49f4-83eb-9b2e8ddf0f11 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:21.233993 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-00c88b40-aa90-4c7f-8551-46a2a0a8f5e1 None None] MFA Rules not processed for user `29d2da8e56cf4360bf0a04bbf055eebd`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:21.237152 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-00c88b40-aa90-4c7f-8551-46a2a0a8f5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:21.376341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba6574-b4ea-49f4-83eb-9b2e8ddf0f11 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:21.422377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572dfee8-4e9b-438f-b5df-2d802b2795bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:21.423938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dfee8-4e9b-438f-b5df-2d802b2795bf admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:21.424291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dfee8-4e9b-438f-b5df-2d802b2795bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:21.424550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dfee8-4e9b-438f-b5df-2d802b2795bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:21.428707 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572dfee8-4e9b-438f-b5df-2d802b2795bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:21.437160 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572dfee8-4e9b-438f-b5df-2d802b2795bf admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:21.468508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 871/1737] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:21.471683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a66260-ae59-4a93-a325-6950be92fafe None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:21.472219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a66260-ae59-4a93-a325-6950be92fafe None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:21.472750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a66260-ae59-4a93-a325-6950be92fafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:21.561017 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 867/1738] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:21 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:21.562810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3a9ae-2d71-43bd-91b6-b2e3803fc124 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:21.562914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-91a66260-ae59-4a93-a325-6950be92fafe None None] MFA Rules not processed for user `dae0a0e651c745a9a9f42e1485832bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:21.565379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-91a66260-ae59-4a93-a325-6950be92fafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:21.578665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3a9ae-2d71-43bd-91b6-b2e3803fc124 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:21.579351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3a9ae-2d71-43bd-91b6-b2e3803fc124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:21.644850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-78e3a9ae-2d71-43bd-91b6-b2e3803fc124 None None] MFA Rules not processed for user `aa222b1c89dc4d2cb1686b4838c7908b`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:21.650942 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-78e3a9ae-2d71-43bd-91b6-b2e3803fc124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:21.749977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 872/1739] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:21.751422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a66260-ae59-4a93-a325-6950be92fafe None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:21.809829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a66260-ae59-4a93-a325-6950be92fafe None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:21.827617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9f69d-f325-4a8a-b111-6b4605e8e8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:21.828764 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9f69d-f325-4a8a-b111-6b4605e8e8a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:21.829282 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9f69d-f325-4a8a-b111-6b4605e8e8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:21.829854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9f69d-f325-4a8a-b111-6b4605e8e8a8 admin admin] PATH_INFO: `/v3/projects/f75052ceff574c0292bfebd9b0b5021b/users/30ba30ef784741fab2f65f3dc139e94e/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:21.865484 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9f69d-f325-4a8a-b111-6b4605e8e8a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f75052ceff574c0292bfebd9b0b5021b, user_id=30ba30ef784741fab2f65f3dc139e94e, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:21.881511 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9f69d-f325-4a8a-b111-6b4605e8e8a8 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:21.898124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 868/1740] 10.222.0.84 () {58 vars in 1015 bytes} [Mon Aug 29 20:27:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:21.902523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3a9ae-2d71-43bd-91b6-b2e3803fc124 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:21.928891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3a9ae-2d71-43bd-91b6-b2e3803fc124 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:21.940772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f1bf5-129b-44d8-a602-e365f0523acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:21.941537 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f1bf5-129b-44d8-a602-e365f0523acc admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:21.941847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f1bf5-129b-44d8-a602-e365f0523acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:21.943656 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f1bf5-129b-44d8-a602-e365f0523acc admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:21.944411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f1bf5-129b-44d8-a602-e365f0523acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:21.951936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f1bf5-129b-44d8-a602-e365f0523acc admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:22.013610 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 873/1741] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:21 2022] PUT /identity/v3/projects/f75052ceff574c0292bfebd9b0b5021b/users/30ba30ef784741fab2f65f3dc139e94e/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:22.016241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab040b2-d28c-4bd1-a161-ccf1d76f9541 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:22.018772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab040b2-d28c-4bd1-a161-ccf1d76f9541 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:22.018772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab040b2-d28c-4bd1-a161-ccf1d76f9541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:22.087996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 869/1742] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:21 2022] POST /identity/v3/projects => generated 369 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:22.089079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f1bf5-129b-44d8-a602-e365f0523acc admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:22.092701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2f7f1bf5-129b-44d8-a602-e365f0523acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:22.167029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-2ab040b2-d28c-4bd1-a161-ccf1d76f9541 None None] MFA Rules not processed for user `30ba30ef784741fab2f65f3dc139e94e`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:22.168415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2ab040b2-d28c-4bd1-a161-ccf1d76f9541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:22.279936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f1bf5-129b-44d8-a602-e365f0523acc admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:22.288132 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6ab64-313a-4464-b850-a6d826bb03c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:22.290563 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6ab64-313a-4464-b850-a6d826bb03c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:22.291719 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6ab64-313a-4464-b850-a6d826bb03c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:22.292261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6ab64-313a-4464-b850-a6d826bb03c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:22.326917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6ab64-313a-4464-b850-a6d826bb03c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:22.328830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6ab64-313a-4464-b850-a6d826bb03c5 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:22.338554 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 874/1743] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:22.341308 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab040b2-d28c-4bd1-a161-ccf1d76f9541 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:22.356823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 870/1744] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:22.357790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6ab64-313a-4464-b850-a6d826bb03c5 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:22.358011 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab040b2-d28c-4bd1-a161-ccf1d76f9541 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:22.361724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c3b6ab64-313a-4464-b850-a6d826bb03c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:22.366749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ad516-b8a5-4959-a240-19354e79b772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:22.367376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ad516-b8a5-4959-a240-19354e79b772 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:22.367744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ad516-b8a5-4959-a240-19354e79b772 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:22.368147 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ad516-b8a5-4959-a240-19354e79b772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:22.378779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ad516-b8a5-4959-a240-19354e79b772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:22.380803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ad516-b8a5-4959-a240-19354e79b772 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:22.398076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 875/1745] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:22.399218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ad516-b8a5-4959-a240-19354e79b772 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:22.453117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:22.516348 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ad516-b8a5-4959-a240-19354e79b772 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:22.523237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a1617-348e-4919-ac69-dbf4419dbbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:22.525825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a1617-348e-4919-ac69-dbf4419dbbbb admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:22.525825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a1617-348e-4919-ac69-dbf4419dbbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:22.525825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a1617-348e-4919-ac69-dbf4419dbbbb admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:22.525825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a1617-348e-4919-ac69-dbf4419dbbbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:22.534605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a1617-348e-4919-ac69-dbf4419dbbbb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:22.594660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6ab64-313a-4464-b850-a6d826bb03c5 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:22.612846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:22.631069 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:22.631361 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:22.631646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] PATH_INFO: `/v3/users/83fa185f035945e8a4943d325b943261` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:22.640411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] RBAC: Authorizing `identity:delete_user(user_id=83fa185f035945e8a4943d325b943261)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:22.643235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:22.654678 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 876/1746] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:22 2022] POST /identity/v3/users => generated 291 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:22.657019 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a1617-348e-4919-ac69-dbf4419dbbbb admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:22.670361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a1617-348e-4919-ac69-dbf4419dbbbb admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:22.680872 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03994a-9828-44f9-b8eb-af65d609c64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:22.681843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03994a-9828-44f9-b8eb-af65d609c64f admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:22.682726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03994a-9828-44f9-b8eb-af65d609c64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:22.685035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03994a-9828-44f9-b8eb-af65d609c64f admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:22.686671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03994a-9828-44f9-b8eb-af65d609c64f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:22.691561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03994a-9828-44f9-b8eb-af65d609c64f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:22.864223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 877/1747] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:22 2022] POST /identity/v3/projects => generated 379 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:22.868277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03994a-9828-44f9-b8eb-af65d609c64f admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:22.963220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03994a-9828-44f9-b8eb-af65d609c64f admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:22.970142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3f9cf-44da-46c7-9b17-c1fa799ec101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:22.971003 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f9cf-44da-46c7-9b17-c1fa799ec101 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:22.971537 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f9cf-44da-46c7-9b17-c1fa799ec101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:22.971913 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f9cf-44da-46c7-9b17-c1fa799ec101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:22.972710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3f9cf-44da-46c7-9b17-c1fa799ec101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:22.974432 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3f9cf-44da-46c7-9b17-c1fa799ec101 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:23.001651 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 878/1748] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:23.001651 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3f9cf-44da-46c7-9b17-c1fa799ec101 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:23.012723 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83fa185f035945e8a4943d325b943261'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:23.015019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83fa185f035945e8a4943d325b943261'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:23.029996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83fa185f035945e8a4943d325b943261'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:23.033084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3f9cf-44da-46c7-9b17-c1fa799ec101 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:23.054347 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c56a7-9d84-4219-b0d4-a42d8673a307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:23.054347 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c56a7-9d84-4219-b0d4-a42d8673a307 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:23.054347 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c56a7-9d84-4219-b0d4-a42d8673a307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:23.054347 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c56a7-9d84-4219-b0d4-a42d8673a307 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:23.055076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c56a7-9d84-4219-b0d4-a42d8673a307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:23.071119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c56a7-9d84-4219-b0d4-a42d8673a307 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:23.138761 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83fa185f035945e8a4943d325b943261'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:23.141259 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83fa185f035945e8a4943d325b943261'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:23.203514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 871/1749] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:22 2022] DELETE /identity/v3/users/83fa185f035945e8a4943d325b943261 => generated 0 bytes in 846 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:23.209485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:23.216056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:23.241722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 879/1750] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:22 2022] POST /identity/v3/users => generated 296 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:23.248375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6cacf8-c5d3-4d34-bf77-3f3c2d4fb004 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:23.248740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6cacf8-c5d3-4d34-bf77-3f3c2d4fb004 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:23.249309 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6cacf8-c5d3-4d34-bf77-3f3c2d4fb004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:23.295716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-cf6cacf8-c5d3-4d34-bf77-3f3c2d4fb004 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:23.297354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cf6cacf8-c5d3-4d34-bf77-3f3c2d4fb004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:23.385493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 880/1751] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:23.386638 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6cacf8-c5d3-4d34-bf77-3f3c2d4fb004 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:23.388277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cf6cacf8-c5d3-4d34-bf77-3f3c2d4fb004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:23.389018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc206341-1cb6-457e-9958-df9837f4533e admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:23.402475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6cacf8-c5d3-4d34-bf77-3f3c2d4fb004 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:23.402673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b4f96-06ee-40b9-8998-6bf9cf604f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:23.403352 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b4f96-06ee-40b9-8998-6bf9cf604f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:23.403991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b4f96-06ee-40b9-8998-6bf9cf604f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:23.404388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b4f96-06ee-40b9-8998-6bf9cf604f8b admin admin] PATH_INFO: `/v3/projects/cb4e8d42e3134e97a9ed23f99cadf104/users/2b3580a62306434bbfeee6cf9ccac53e/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:23.410485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c60468-02e2-4221-a8c4-e006c0ccbbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:23.411101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c60468-02e2-4221-a8c4-e006c0ccbbb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:23.411380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c60468-02e2-4221-a8c4-e006c0ccbbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:23.411701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c60468-02e2-4221-a8c4-e006c0ccbbb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:23.412503 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c60468-02e2-4221-a8c4-e006c0ccbbb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:23.414220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c60468-02e2-4221-a8c4-e006c0ccbbb8 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:23.441393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 881/1752] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:23.442404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c60468-02e2-4221-a8c4-e006c0ccbbb8 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:23.445349 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b4f96-06ee-40b9-8998-6bf9cf604f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb4e8d42e3134e97a9ed23f99cadf104, user_id=2b3580a62306434bbfeee6cf9ccac53e, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:23.455109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-14c60468-02e2-4221-a8c4-e006c0ccbbb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:23.456933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b4f96-06ee-40b9-8998-6bf9cf604f8b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:23.552624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:23.557141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 872/1753] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:23 2022] PUT /identity/v3/projects/cb4e8d42e3134e97a9ed23f99cadf104/users/2b3580a62306434bbfeee6cf9ccac53e/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:23.558547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b4f96-06ee-40b9-8998-6bf9cf604f8b admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:23.577523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c60468-02e2-4221-a8c4-e006c0ccbbb8 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:23.644899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b4f96-06ee-40b9-8998-6bf9cf604f8b admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:23.654620 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813aa5c1-87ea-4bf5-a817-cb87adec806c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:23.655738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813aa5c1-87ea-4bf5-a817-cb87adec806c admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:23.656472 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813aa5c1-87ea-4bf5-a817-cb87adec806c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:23.657019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813aa5c1-87ea-4bf5-a817-cb87adec806c admin admin] PATH_INFO: `/v3/projects/a3cccc301a2f4d47a1df698442b7b240/users/feba9584d03c46e2937cacdf57e2d8d4/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:23.688637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813aa5c1-87ea-4bf5-a817-cb87adec806c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cccc301a2f4d47a1df698442b7b240, user_id=feba9584d03c46e2937cacdf57e2d8d4, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:23.695435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813aa5c1-87ea-4bf5-a817-cb87adec806c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:23.717340 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45c748-e081-4603-9f6a-b918dc919970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:23.718827 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45c748-e081-4603-9f6a-b918dc919970 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:23.719360 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45c748-e081-4603-9f6a-b918dc919970 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:23.719764 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45c748-e081-4603-9f6a-b918dc919970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:23.733432 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45c748-e081-4603-9f6a-b918dc919970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:23.734990 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45c748-e081-4603-9f6a-b918dc919970 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:23.747471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 882/1754] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:23.748435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45c748-e081-4603-9f6a-b918dc919970 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:23.758926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45c748-e081-4603-9f6a-b918dc919970 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:23.764292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ffbd7-4cca-4661-a819-ab9bee1361e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:23.764969 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ffbd7-4cca-4661-a819-ab9bee1361e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:23.765288 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ffbd7-4cca-4661-a819-ab9bee1361e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:23.765472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ffbd7-4cca-4661-a819-ab9bee1361e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:23.766116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ffbd7-4cca-4661-a819-ab9bee1361e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:23.768072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ffbd7-4cca-4661-a819-ab9bee1361e1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:23.784442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 883/1755] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:23.785381 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ffbd7-4cca-4661-a819-ab9bee1361e1 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:23.797838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ffbd7-4cca-4661-a819-ab9bee1361e1 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:23.803163 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810a7db-9d50-4dc8-a5cb-204dcb1de6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:23.803839 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810a7db-9d50-4dc8-a5cb-204dcb1de6aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:23.804470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810a7db-9d50-4dc8-a5cb-204dcb1de6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:23.805516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810a7db-9d50-4dc8-a5cb-204dcb1de6aa admin admin] PATH_INFO: `/v3/projects/cb4e8d42e3134e97a9ed23f99cadf104/users/2b3580a62306434bbfeee6cf9ccac53e/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:23.808085 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810a7db-9d50-4dc8-a5cb-204dcb1de6aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb4e8d42e3134e97a9ed23f99cadf104, user_id=2b3580a62306434bbfeee6cf9ccac53e, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:23.811208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810a7db-9d50-4dc8-a5cb-204dcb1de6aa admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:23.842155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 884/1756] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:23 2022] PUT /identity/v3/projects/cb4e8d42e3134e97a9ed23f99cadf104/users/2b3580a62306434bbfeee6cf9ccac53e/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:23.852345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a2c15-8222-4245-94fd-998f1444b026 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:23.853309 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a2c15-8222-4245-94fd-998f1444b026 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:23.853637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a2c15-8222-4245-94fd-998f1444b026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:23.883643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 873/1757] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:23 2022] PUT /identity/v3/projects/a3cccc301a2f4d47a1df698442b7b240/users/feba9584d03c46e2937cacdf57e2d8d4/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:23.892832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813aa5c1-87ea-4bf5-a817-cb87adec806c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:23.930421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-0f3a2c15-8222-4245-94fd-998f1444b026 None None] MFA Rules not processed for user `2b3580a62306434bbfeee6cf9ccac53e`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:23.932517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-0f3a2c15-8222-4245-94fd-998f1444b026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:24.058788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 885/1758] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:24.059079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813aa5c1-87ea-4bf5-a817-cb87adec806c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:24.072075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a2c15-8222-4245-94fd-998f1444b026 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:24.075188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-0f3a2c15-8222-4245-94fd-998f1444b026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:24.075334 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7e038-7310-49cf-bd1c-edfde18af949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:24.076035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7e038-7310-49cf-bd1c-edfde18af949 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:24.076326 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7e038-7310-49cf-bd1c-edfde18af949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:24.076724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7e038-7310-49cf-bd1c-edfde18af949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:24.077308 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7e038-7310-49cf-bd1c-edfde18af949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:24.078936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7e038-7310-49cf-bd1c-edfde18af949 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:24.108705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 874/1759] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:27:24.112796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7e038-7310-49cf-bd1c-edfde18af949 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:24.128413 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7e038-7310-49cf-bd1c-edfde18af949 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:24.140854 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4b145-6e41-4daf-9134-82e3cf720d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:24.143504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4b145-6e41-4daf-9134-82e3cf720d31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:24.143940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4b145-6e41-4daf-9134-82e3cf720d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:24.144240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4b145-6e41-4daf-9134-82e3cf720d31 admin admin] PATH_INFO: `/v3/projects/a3cccc301a2f4d47a1df698442b7b240/users/feba9584d03c46e2937cacdf57e2d8d4/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:24.149014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4b145-6e41-4daf-9134-82e3cf720d31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cccc301a2f4d47a1df698442b7b240, user_id=feba9584d03c46e2937cacdf57e2d8d4, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:24.151432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4b145-6e41-4daf-9134-82e3cf720d31 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:24.179398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 875/1760] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:24 2022] PUT /identity/v3/projects/a3cccc301a2f4d47a1df698442b7b240/users/feba9584d03c46e2937cacdf57e2d8d4/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:24.195101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057971b3-2011-4748-a98b-12afa2d3eab4 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:24.195430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057971b3-2011-4748-a98b-12afa2d3eab4 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:24.195638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057971b3-2011-4748-a98b-12afa2d3eab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:24.226245 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a2c15-8222-4245-94fd-998f1444b026 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:24.246929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:24.268232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-057971b3-2011-4748-a98b-12afa2d3eab4 None None] MFA Rules not processed for user `feba9584d03c46e2937cacdf57e2d8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:24.269527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-057971b3-2011-4748-a98b-12afa2d3eab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:24.406028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aba405-8028-4afd-a221-23df77fe0e44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:24.406693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aba405-8028-4afd-a221-23df77fe0e44 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:24.406998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aba405-8028-4afd-a221-23df77fe0e44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:24.407291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aba405-8028-4afd-a221-23df77fe0e44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:24.415082 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 876/1761] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:24.422793 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a2a68-eb13-492e-a9b9-99081c9a91b4 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:24.423081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a2a68-eb13-492e-a9b9-99081c9a91b4 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:24.423359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a2a68-eb13-492e-a9b9-99081c9a91b4 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:24.425024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 877/1762] 10.222.0.85 () {54 vars in 921 bytes} [Mon Aug 29 20:27:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:24.503846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a2a68-eb13-492e-a9b9-99081c9a91b4 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:24.507001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b96a2a68-eb13-492e-a9b9-99081c9a91b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:24.507627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aba405-8028-4afd-a221-23df77fe0e44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:24.509250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aba405-8028-4afd-a221-23df77fe0e44 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:24.524066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 886/1763] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:24.688449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a2a68-eb13-492e-a9b9-99081c9a91b4 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:24.703139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:24.703139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:24.703139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:24.703139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] PATH_INFO: `/v3/projects/bad47a00811647e39f59ff4e0b6bd4ce` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:24.703826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad47a00811647e39f59ff4e0b6bd4ce)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:24.703826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:24.748954 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aba405-8028-4afd-a221-23df77fe0e44 service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:24.758013 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aba405-8028-4afd-a221-23df77fe0e44 service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:24.762708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611334f-088b-4e57-bfed-e06a95db503b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:24.763406 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611334f-088b-4e57-bfed-e06a95db503b service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:24.763763 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611334f-088b-4e57-bfed-e06a95db503b service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:24.764100 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611334f-088b-4e57-bfed-e06a95db503b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:24.776204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611334f-088b-4e57-bfed-e06a95db503b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:24.778350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611334f-088b-4e57-bfed-e06a95db503b service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:24.790862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 887/1764] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:27:24.803238 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f0952-e91c-4761-9c12-4a59859ab959 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:24.803964 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f0952-e91c-4761-9c12-4a59859ab959 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:24.805888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f0952-e91c-4761-9c12-4a59859ab959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:24.865395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-972f0952-e91c-4761-9c12-4a59859ab959 None None] MFA Rules not processed for user `5288af2dbc15438c8b209b0ba2454cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:24.867578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-972f0952-e91c-4761-9c12-4a59859ab959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:24.910589 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad47a00811647e39f59ff4e0b6bd4ce', 'request_id': 'req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:24.920555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad47a00811647e39f59ff4e0b6bd4ce', 'request_id': 'req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:25.029256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad47a00811647e39f59ff4e0b6bd4ce', 'request_id': 'req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:25.062603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] The token cache is being invalidate because project bad47a00811647e39f59ff4e0b6bd4ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:25.062896 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:25.063694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:25.065666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 878/1765] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:24 2022] DELETE /identity/v3/projects/bad47a00811647e39f59ff4e0b6bd4ce => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:25.070274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:25.074701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:25.079436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 888/1766] 10.222.0.85 () {58 vars in 1018 bytes} [Mon Aug 29 20:27:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:25.098546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:25.126376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cd3c6-fb44-4bcb-8335-26b4307493bf None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:25.126627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cd3c6-fb44-4bcb-8335-26b4307493bf None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:25.126946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cd3c6-fb44-4bcb-8335-26b4307493bf None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:25.129512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 889/1767] 10.222.0.85 () {54 vars in 961 bytes} [Mon Aug 29 20:27:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:27:25.137799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1639ec-d9ab-4562-b8c9-ee4db810001d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:25.138992 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1639ec-d9ab-4562-b8c9-ee4db810001d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:25.139427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1639ec-d9ab-4562-b8c9-ee4db810001d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:25.232139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-cf1639ec-d9ab-4562-b8c9-ee4db810001d None None] MFA Rules not processed for user `5288af2dbc15438c8b209b0ba2454cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:25.264267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cf1639ec-d9ab-4562-b8c9-ee4db810001d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:25.284492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 890/1768] 10.222.0.85 () {58 vars in 1058 bytes} [Mon Aug 29 20:27:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:25.288158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ca12b-d2b8-4a65-a5f9-55f3da5e75a6 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:25.293453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1639ec-d9ab-4562-b8c9-ee4db810001d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:25.303213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e35a88-c809-43f9-a968-cb6912aad2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:25.303952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e35a88-c809-43f9-a968-cb6912aad2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:25.304255 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e35a88-c809-43f9-a968-cb6912aad2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:25.304532 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e35a88-c809-43f9-a968-cb6912aad2fc admin admin] PATH_INFO: `/v3/users/581c78daeb264cf4be5b0269c3301db9` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:25.320348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e35a88-c809-43f9-a968-cb6912aad2fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=581c78daeb264cf4be5b0269c3301db9)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:25.321114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1639ec-d9ab-4562-b8c9-ee4db810001d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:25.329973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e35a88-c809-43f9-a968-cb6912aad2fc admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:25.338084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825da80-fb4a-4e2e-9766-052bcc1367c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:25.341465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825da80-fb4a-4e2e-9766-052bcc1367c1 service placement] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:25.341926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825da80-fb4a-4e2e-9766-052bcc1367c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:25.343244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825da80-fb4a-4e2e-9766-052bcc1367c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:25.365443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825da80-fb4a-4e2e-9766-052bcc1367c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:25.367570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825da80-fb4a-4e2e-9766-052bcc1367c1 service placement] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:25.379926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 891/1769] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:27:25.635430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-83e35a88-c809-43f9-a968-cb6912aad2fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '581c78daeb264cf4be5b0269c3301db9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:25.644785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-83e35a88-c809-43f9-a968-cb6912aad2fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '581c78daeb264cf4be5b0269c3301db9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:25.678448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-83e35a88-c809-43f9-a968-cb6912aad2fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '581c78daeb264cf4be5b0269c3301db9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:25.795273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-83e35a88-c809-43f9-a968-cb6912aad2fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '581c78daeb264cf4be5b0269c3301db9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:25.795922 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-83e35a88-c809-43f9-a968-cb6912aad2fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '581c78daeb264cf4be5b0269c3301db9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:25.845241 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 879/1770] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:25 2022] DELETE /identity/v3/users/581c78daeb264cf4be5b0269c3301db9 => generated 0 bytes in 772 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:26.789935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817188ec-6a03-43d8-b5e5-7a69f7948ead None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:26.790542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817188ec-6a03-43d8-b5e5-7a69f7948ead None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:26.790542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817188ec-6a03-43d8-b5e5-7a69f7948ead None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:26.793588 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 892/1771] 10.222.0.85 () {54 vars in 921 bytes} [Mon Aug 29 20:27:26 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:26.811247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058a029-1208-4cd2-ac18-96b1d113dc5e None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:26.811843 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058a029-1208-4cd2-ac18-96b1d113dc5e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:26.812013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058a029-1208-4cd2-ac18-96b1d113dc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:26.876215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-3058a029-1208-4cd2-ac18-96b1d113dc5e None None] MFA Rules not processed for user `5288af2dbc15438c8b209b0ba2454cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:26.877765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3058a029-1208-4cd2-ac18-96b1d113dc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:27.004278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817188ec-6a03-43d8-b5e5-7a69f7948ead None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:27.006361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-817188ec-6a03-43d8-b5e5-7a69f7948ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:27.007689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 880/1772] 10.222.0.85 () {58 vars in 1018 bytes} [Mon Aug 29 20:27:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:27.098811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058a029-1208-4cd2-ac18-96b1d113dc5e None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:27.106326 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3058a029-1208-4cd2-ac18-96b1d113dc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:27.142810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058a029-1208-4cd2-ac18-96b1d113dc5e None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:27.159043 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5125a-c1ca-4006-b18f-80d84b380703 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:27.161741 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5125a-c1ca-4006-b18f-80d84b380703 service placement] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:27.163136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5125a-c1ca-4006-b18f-80d84b380703 service placement] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:27.164060 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5125a-c1ca-4006-b18f-80d84b380703 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:27.183718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817188ec-6a03-43d8-b5e5-7a69f7948ead None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:27.186079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5125a-c1ca-4006-b18f-80d84b380703 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:27.188968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d729c-785d-49a9-8b25-218bfc56ef00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:27.189271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d729c-785d-49a9-8b25-218bfc56ef00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:27.189547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d729c-785d-49a9-8b25-218bfc56ef00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:27.189858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d729c-785d-49a9-8b25-218bfc56ef00 admin admin] PATH_INFO: `/v3/users/8b0cd78e77ee4daab447eba771307ff6` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:27.190453 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5125a-c1ca-4006-b18f-80d84b380703 service placement] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:27.191289 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d729c-785d-49a9-8b25-218bfc56ef00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b0cd78e77ee4daab447eba771307ff6)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:27.194255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d729c-785d-49a9-8b25-218bfc56ef00 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:27.291099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 881/1773] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:27.293186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5125a-c1ca-4006-b18f-80d84b380703 service placement] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:27.299952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8ff5125a-c1ca-4006-b18f-80d84b380703 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:27.479772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-0e8d729c-785d-49a9-8b25-218bfc56ef00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b0cd78e77ee4daab447eba771307ff6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:27.507801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-0e8d729c-785d-49a9-8b25-218bfc56ef00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b0cd78e77ee4daab447eba771307ff6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:27.543562 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:27.556928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5125a-c1ca-4006-b18f-80d84b380703 service placement] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:27.576615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24558a67-c01b-4df9-bc3e-d04af830df6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feba9584d03c46e2937cacdf57e2d8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cccc301a2f4d47a1df698442b7b240', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:27.577613 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24558a67-c01b-4df9-bc3e-d04af830df6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:27.578045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24558a67-c01b-4df9-bc3e-d04af830df6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:27.578487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24558a67-c01b-4df9-bc3e-d04af830df6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:27.580132 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24558a67-c01b-4df9-bc3e-d04af830df6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:27.580896 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24558a67-c01b-4df9-bc3e-d04af830df6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:27.582541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24558a67-c01b-4df9-bc3e-d04af830df6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:27.595945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:27.600148 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-0e8d729c-785d-49a9-8b25-218bfc56ef00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b0cd78e77ee4daab447eba771307ff6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:27.601087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-0e8d729c-785d-49a9-8b25-218bfc56ef00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b0cd78e77ee4daab447eba771307ff6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:27.634785 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-0e8d729c-785d-49a9-8b25-218bfc56ef00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b0cd78e77ee4daab447eba771307ff6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:27.653233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 893/1774] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:27 2022] DELETE /identity/v3/users/8b0cd78e77ee4daab447eba771307ff6 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:27.657087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbd840-6134-46aa-a575-cb82d9803c34 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:27.657378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbd840-6134-46aa-a575-cb82d9803c34 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:27.657655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbd840-6134-46aa-a575-cb82d9803c34 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:27.661244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 894/1775] 10.222.0.85 () {54 vars in 921 bytes} [Mon Aug 29 20:27:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:27.661901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 882/1776] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:27.662215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbd840-6134-46aa-a575-cb82d9803c34 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:27.663322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24558a67-c01b-4df9-bc3e-d04af830df6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:27.672233 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-24558a67-c01b-4df9-bc3e-d04af830df6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:27.673669 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-0bbbd840-6134-46aa-a575-cb82d9803c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:27.733655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24558a67-c01b-4df9-bc3e-d04af830df6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:27.901711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbd840-6134-46aa-a575-cb82d9803c34 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:27.913884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:27.915850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:27.918893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:27.918893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] PATH_INFO: `/v3/projects/057454e7c1b74b2fb61ce703b1484f84` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:27.918893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=057454e7c1b74b2fb61ce703b1484f84)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:27.927687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:27.979964 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894de00b-d08c-4746-acc5-e63e0046d623 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feba9584d03c46e2937cacdf57e2d8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cccc301a2f4d47a1df698442b7b240', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:27.982403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894de00b-d08c-4746-acc5-e63e0046d623 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:27.983290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894de00b-d08c-4746-acc5-e63e0046d623 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:27.984227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894de00b-d08c-4746-acc5-e63e0046d623 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:27.985776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894de00b-d08c-4746-acc5-e63e0046d623 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:27.988457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894de00b-d08c-4746-acc5-e63e0046d623 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:28.103526 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 883/1777] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:27 2022] POST /identity/v3/projects => generated 354 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:28.109270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58688ae-0fa7-41c7-9e44-e2c40113027d None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:28.109808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58688ae-0fa7-41c7-9e44-e2c40113027d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:28.110293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58688ae-0fa7-41c7-9e44-e2c40113027d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:28.177312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '057454e7c1b74b2fb61ce703b1484f84', 'request_id': 'req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:28.189625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '057454e7c1b74b2fb61ce703b1484f84', 'request_id': 'req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:28.197475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f58688ae-0fa7-41c7-9e44-e2c40113027d None None] MFA Rules not processed for user `805df3047fee45c7a702d72a5c082eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:28.202562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '057454e7c1b74b2fb61ce703b1484f84', 'request_id': 'req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:28.202696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f58688ae-0fa7-41c7-9e44-e2c40113027d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:28.277174 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] The token cache is being invalidate because project 057454e7c1b74b2fb61ce703b1484f84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:28.277174 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:28.281694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:28.284598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 895/1778] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:27 2022] DELETE /identity/v3/projects/057454e7c1b74b2fb61ce703b1484f84 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:28.285639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:28.298000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:28.348052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 884/1779] 10.222.0.85 () {58 vars in 1018 bytes} [Mon Aug 29 20:27:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:28.357876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a178cf-107f-406e-9e81-73d590d40ff6 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:28.358722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a178cf-107f-406e-9e81-73d590d40ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:28.361248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a178cf-107f-406e-9e81-73d590d40ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:28.395430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-e0a178cf-107f-406e-9e81-73d590d40ff6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:28.397025 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e0a178cf-107f-406e-9e81-73d590d40ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:28.441787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fad1fb-bc0e-4570-b952-4c3ebf84ef30 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:28.539443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 885/1780] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:28.539443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a178cf-107f-406e-9e81-73d590d40ff6 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:28.544553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e0a178cf-107f-406e-9e81-73d590d40ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:28.575664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a178cf-107f-406e-9e81-73d590d40ff6 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:28.582193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615b4f2-3aeb-4cbc-b18a-ce9e5db06a6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feba9584d03c46e2937cacdf57e2d8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cccc301a2f4d47a1df698442b7b240', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:28.582920 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615b4f2-3aeb-4cbc-b18a-ce9e5db06a6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:28.583279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615b4f2-3aeb-4cbc-b18a-ce9e5db06a6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:28.583729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615b4f2-3aeb-4cbc-b18a-ce9e5db06a6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:28.584620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9615b4f2-3aeb-4cbc-b18a-ce9e5db06a6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:28.585075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9615b4f2-3aeb-4cbc-b18a-ce9e5db06a6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:28.587870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:28.588091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9615b4f2-3aeb-4cbc-b18a-ce9e5db06a6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:28.588584 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:28.588899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:28.589209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] PATH_INFO: `/v3/projects/709e855dc6b0428d84a8f2cfeb7df063` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:28.590354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] RBAC: Authorizing `identity:delete_project(project_id=709e855dc6b0428d84a8f2cfeb7df063)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:28.592335 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:28.618049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 896/1781] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:28.619991 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615b4f2-3aeb-4cbc-b18a-ce9e5db06a6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:28.632412 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9615b4f2-3aeb-4cbc-b18a-ce9e5db06a6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:28.732345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615b4f2-3aeb-4cbc-b18a-ce9e5db06a6c tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:28.745774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795046d-8ec5-403f-8dcd-9b3dc490846f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:28.746399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795046d-8ec5-403f-8dcd-9b3dc490846f service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:28.747099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795046d-8ec5-403f-8dcd-9b3dc490846f service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:28.747099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795046d-8ec5-403f-8dcd-9b3dc490846f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:28.776361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795046d-8ec5-403f-8dcd-9b3dc490846f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:28.778602 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795046d-8ec5-403f-8dcd-9b3dc490846f service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:28.821124 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 897/1782] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:28.822653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8632504-b3d0-4758-8a85-e155f6a821f9 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:28.822937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8632504-b3d0-4758-8a85-e155f6a821f9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:28.823237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8632504-b3d0-4758-8a85-e155f6a821f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:28.829186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '709e855dc6b0428d84a8f2cfeb7df063', 'request_id': 'req-e566e146-994a-4d4b-ade3-bd2b20a7392d'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:28.838922 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '709e855dc6b0428d84a8f2cfeb7df063', 'request_id': 'req-e566e146-994a-4d4b-ade3-bd2b20a7392d'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:28.904633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-c8632504-b3d0-4758-8a85-e155f6a821f9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:28.904633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c8632504-b3d0-4758-8a85-e155f6a821f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:28.978401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 898/1783] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:29.002064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8632504-b3d0-4758-8a85-e155f6a821f9 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:29.088725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8632504-b3d0-4758-8a85-e155f6a821f9 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:29.094294 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feba9584d03c46e2937cacdf57e2d8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cccc301a2f4d47a1df698442b7b240', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:29.094967 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:29.095254 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:29.095530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] PATH_INFO: `/v3/projects/53b768b04ed048bd9eb73738b2b8a848` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:29.096852 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: Authorizing `identity:delete_project(project_id=53b768b04ed048bd9eb73738b2b8a848)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:29.098865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:30.508887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '709e855dc6b0428d84a8f2cfeb7df063', 'request_id': 'req-e566e146-994a-4d4b-ade3-bd2b20a7392d'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:30.520946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] The token cache is being invalidate because project 709e855dc6b0428d84a8f2cfeb7df063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:30.521241 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:30.521910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:30.523400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 886/1784] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:28 2022] DELETE /identity/v3/projects/709e855dc6b0428d84a8f2cfeb7df063 => generated 0 bytes in 1991 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:30.524199 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:30.525885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:30.565473 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566e146-994a-4d4b-ade3-bd2b20a7392d admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:30.574064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53b768b04ed048bd9eb73738b2b8a848', 'request_id': 'req-184d9cd3-76ba-420e-a629-e0971a4bab47'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:30.584848 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53b768b04ed048bd9eb73738b2b8a848', 'request_id': 'req-184d9cd3-76ba-420e-a629-e0971a4bab47'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:30.600198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53b768b04ed048bd9eb73738b2b8a848', 'request_id': 'req-184d9cd3-76ba-420e-a629-e0971a4bab47'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:30.688647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f59868-c7d0-43c1-93dc-47e1058bfa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:30.689752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f59868-c7d0-43c1-93dc-47e1058bfa04 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:30.690224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f59868-c7d0-43c1-93dc-47e1058bfa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:30.700826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] The token cache is being invalidate because project 53b768b04ed048bd9eb73738b2b8a848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:30.703599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:30.704551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f59868-c7d0-43c1-93dc-47e1058bfa04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:30.706577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f59868-c7d0-43c1-93dc-47e1058bfa04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:30.713878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:30.716763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f59868-c7d0-43c1-93dc-47e1058bfa04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:30.717917 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 899/1785] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:28 2022] DELETE /identity/v3/projects/53b768b04ed048bd9eb73738b2b8a848 => generated 0 bytes in 1738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:30.718914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:30.721554 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:30.732694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f59868-c7d0-43c1-93dc-47e1058bfa04 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:30.751863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d9cd3-76ba-420e-a629-e0971a4bab47 tempest-VolumeQuotaClassesTest-599541210 tempest-VolumeQuotaClassesTest-599541210-project-admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:30.751863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:30.752047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 887/1786] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:30.752136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:30.752136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:30.753415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f59868-c7d0-43c1-93dc-47e1058bfa04 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:30.754919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] PATH_INFO: `/v3/users/29d2da8e56cf4360bf0a04bbf055eebd` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:30.756448 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29d2da8e56cf4360bf0a04bbf055eebd)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:30.756578 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a6f59868-c7d0-43c1-93dc-47e1058bfa04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:30.759121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:30.779841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f59868-c7d0-43c1-93dc-47e1058bfa04 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:30.795337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73d643-d618-4b92-b818-11a144051b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:30.796081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73d643-d618-4b92-b818-11a144051b79 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:30.796514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73d643-d618-4b92-b818-11a144051b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:30.797002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73d643-d618-4b92-b818-11a144051b79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:30.797750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73d643-d618-4b92-b818-11a144051b79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:30.801086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73d643-d618-4b92-b818-11a144051b79 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:30.982483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 888/1787] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:30 2022] POST /identity/v3/projects => generated 365 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:30.983510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73d643-d618-4b92-b818-11a144051b79 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:31.033930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29d2da8e56cf4360bf0a04bbf055eebd'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:31.057204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29d2da8e56cf4360bf0a04bbf055eebd'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:31.098540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73d643-d618-4b92-b818-11a144051b79 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:31.108346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45babe07-04b4-4240-ab99-497283ca1f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:31.108346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45babe07-04b4-4240-ab99-497283ca1f3e admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:31.108346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45babe07-04b4-4240-ab99-497283ca1f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:31.108346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45babe07-04b4-4240-ab99-497283ca1f3e admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:31.108908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45babe07-04b4-4240-ab99-497283ca1f3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:31.108908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45babe07-04b4-4240-ab99-497283ca1f3e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:31.196252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29d2da8e56cf4360bf0a04bbf055eebd'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:31.197056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29d2da8e56cf4360bf0a04bbf055eebd'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:31.212123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29d2da8e56cf4360bf0a04bbf055eebd'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:31.218519 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 900/1788] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:30 2022] DELETE /identity/v3/users/29d2da8e56cf4360bf0a04bbf055eebd => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:31.242607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 889/1789] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:30 2022] POST /identity/v3/users => generated 283 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:31.249244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:31.252976 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:31.351536 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21bbca-41a7-4096-83f5-894a549ddf23 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:31.356825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020f71d-ca83-4179-ae53-68538884cd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:31.358227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020f71d-ca83-4179-ae53-68538884cd39 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:31.358227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020f71d-ca83-4179-ae53-68538884cd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:31.358648 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020f71d-ca83-4179-ae53-68538884cd39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:31.359718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020f71d-ca83-4179-ae53-68538884cd39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:31.361721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020f71d-ca83-4179-ae53-68538884cd39 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:31.377757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 901/1790] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:31.379036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45babe07-04b4-4240-ab99-497283ca1f3e admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:31.391044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45babe07-04b4-4240-ab99-497283ca1f3e admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:31.398214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f58bf-421b-4591-a633-70b6fa048150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:31.398214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f58bf-421b-4591-a633-70b6fa048150 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:31.398588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f58bf-421b-4591-a633-70b6fa048150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:31.398821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f58bf-421b-4591-a633-70b6fa048150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:31.399543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f58bf-421b-4591-a633-70b6fa048150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:31.401349 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f58bf-421b-4591-a633-70b6fa048150 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:31.417516 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 890/1791] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:27:31.424084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020f71d-ca83-4179-ae53-68538884cd39 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:31.456315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020f71d-ca83-4179-ae53-68538884cd39 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:31.465109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629e926-d5a1-4b81-8c50-c03720c6c620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:31.466358 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629e926-d5a1-4b81-8c50-c03720c6c620 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:31.466836 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629e926-d5a1-4b81-8c50-c03720c6c620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:31.467855 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629e926-d5a1-4b81-8c50-c03720c6c620 admin admin] PATH_INFO: `/v3/projects/504ac0aeb5c64252b3c39fb31254cffc/users/6263de5280c64c5bb1637ca7b95ea501/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:31.500880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629e926-d5a1-4b81-8c50-c03720c6c620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=504ac0aeb5c64252b3c39fb31254cffc, user_id=6263de5280c64c5bb1637ca7b95ea501, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:31.504854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:31.507184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629e926-d5a1-4b81-8c50-c03720c6c620 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:31.599377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f58bf-421b-4591-a633-70b6fa048150 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:31.601640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f40f58bf-421b-4591-a633-70b6fa048150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:31.621787 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f58bf-421b-4591-a633-70b6fa048150 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:31.628979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 902/1792] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:31 2022] PUT /identity/v3/projects/504ac0aeb5c64252b3c39fb31254cffc/users/6263de5280c64c5bb1637ca7b95ea501/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:31.629827 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629e926-d5a1-4b81-8c50-c03720c6c620 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:31.646345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-1629e926-d5a1-4b81-8c50-c03720c6c620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:31.723118 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70da1a7-601c-4f9d-ae1c-bdb553c55b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:31.723555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70da1a7-601c-4f9d-ae1c-bdb553c55b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:31.723924 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70da1a7-601c-4f9d-ae1c-bdb553c55b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:31.724649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70da1a7-601c-4f9d-ae1c-bdb553c55b08 admin admin] PATH_INFO: `/v3/users/dae0a0e651c745a9a9f42e1485832bc9` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:31.727014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70da1a7-601c-4f9d-ae1c-bdb553c55b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae0a0e651c745a9a9f42e1485832bc9)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:31.730444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70da1a7-601c-4f9d-ae1c-bdb553c55b08 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:31.738242 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629e926-d5a1-4b81-8c50-c03720c6c620 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:31.743671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:31.744327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:31.744606 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:31.744877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] PATH_INFO: `/v3/projects/8ce8eadad3854460a539325e16b35d26` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:31.746021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ce8eadad3854460a539325e16b35d26)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:31.748043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:32.037838 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b70da1a7-601c-4f9d-ae1c-bdb553c55b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae0a0e651c745a9a9f42e1485832bc9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.038342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ce8eadad3854460a539325e16b35d26', 'request_id': 'req-a2fc1e94-9a82-44c2-97c2-42c5803a199a'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.038739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b70da1a7-601c-4f9d-ae1c-bdb553c55b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae0a0e651c745a9a9f42e1485832bc9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.045163 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ce8eadad3854460a539325e16b35d26', 'request_id': 'req-a2fc1e94-9a82-44c2-97c2-42c5803a199a'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.046206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b70da1a7-601c-4f9d-ae1c-bdb553c55b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae0a0e651c745a9a9f42e1485832bc9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.051486 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ce8eadad3854460a539325e16b35d26', 'request_id': 'req-a2fc1e94-9a82-44c2-97c2-42c5803a199a'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.166294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b70da1a7-601c-4f9d-ae1c-bdb553c55b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae0a0e651c745a9a9f42e1485832bc9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.166972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b70da1a7-601c-4f9d-ae1c-bdb553c55b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae0a0e651c745a9a9f42e1485832bc9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.191112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 891/1793] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:31 2022] DELETE /identity/v3/users/dae0a0e651c745a9a9f42e1485832bc9 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:32.192615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d166f6-ce9e-4e0a-aa90-f6379668fb40 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:32.193005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d166f6-ce9e-4e0a-aa90-f6379668fb40 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:32.194025 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d166f6-ce9e-4e0a-aa90-f6379668fb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.225599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-b5d166f6-ce9e-4e0a-aa90-f6379668fb40 None None] MFA Rules not processed for user `6263de5280c64c5bb1637ca7b95ea501`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:32.226821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b5d166f6-ce9e-4e0a-aa90-f6379668fb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:32.274316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] The token cache is being invalidate because project 8ce8eadad3854460a539325e16b35d26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:32.274611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.275255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.277630 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 903/1794] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:31 2022] DELETE /identity/v3/projects/8ce8eadad3854460a539325e16b35d26 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:32.278450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:32.279946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:32.293302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 892/1795] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:32.294608 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986d550-519e-4151-8afd-a57a323cc3a7 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:32.294894 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986d550-519e-4151-8afd-a57a323cc3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:32.296663 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986d550-519e-4151-8afd-a57a323cc3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.314732 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-5986d550-519e-4151-8afd-a57a323cc3a7 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:32.315929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5986d550-519e-4151-8afd-a57a323cc3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:32.348083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc1e94-9a82-44c2-97c2-42c5803a199a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:32.374786 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:32.377376 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 893/1796] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:32.378625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f946f78-46b0-4732-b729-b366b51333f6 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:32.378985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f946f78-46b0-4732-b729-b366b51333f6 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:32.379344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f946f78-46b0-4732-b729-b366b51333f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.399255 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-5f946f78-46b0-4732-b729-b366b51333f6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:32.405541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5f946f78-46b0-4732-b729-b366b51333f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:32.416286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 894/1797] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:32.416286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f946f78-46b0-4732-b729-b366b51333f6 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:32.416286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5f946f78-46b0-4732-b729-b366b51333f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:32.418174 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dead5-f61f-47af-b53f-60e4ae64feb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:32.418839 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dead5-f61f-47af-b53f-60e4ae64feb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:32.419189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dead5-f61f-47af-b53f-60e4ae64feb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:32.419466 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dead5-f61f-47af-b53f-60e4ae64feb2 admin admin] PATH_INFO: `/v3/users/30ba30ef784741fab2f65f3dc139e94e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.420625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942dead5-f61f-47af-b53f-60e4ae64feb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ba30ef784741fab2f65f3dc139e94e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:32.422766 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942dead5-f61f-47af-b53f-60e4ae64feb2 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:32.423029 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f946f78-46b0-4732-b729-b366b51333f6 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:32.428858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2a745-37e6-4f92-8f72-8423418cd55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:32.429106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2a745-37e6-4f92-8f72-8423418cd55d admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:32.429562 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2a745-37e6-4f92-8f72-8423418cd55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:32.436516 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2a745-37e6-4f92-8f72-8423418cd55d admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.436516 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2a745-37e6-4f92-8f72-8423418cd55d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:32.436516 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2a745-37e6-4f92-8f72-8423418cd55d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:32.562762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 895/1798] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:32 2022] POST /identity/v3/projects => generated 363 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:32.563840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2a745-37e6-4f92-8f72-8423418cd55d admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:32.565498 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-52c2a745-37e6-4f92-8f72-8423418cd55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:32.624763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2a745-37e6-4f92-8f72-8423418cd55d admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:32.629083 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63b648-4b48-4a10-bc2e-1f636c07fa4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:32.629687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63b648-4b48-4a10-bc2e-1f636c07fa4b service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:32.629971 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63b648-4b48-4a10-bc2e-1f636c07fa4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:32.630398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63b648-4b48-4a10-bc2e-1f636c07fa4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.638007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-942dead5-f61f-47af-b53f-60e4ae64feb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ba30ef784741fab2f65f3dc139e94e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.645114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-942dead5-f61f-47af-b53f-60e4ae64feb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ba30ef784741fab2f65f3dc139e94e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.690076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63b648-4b48-4a10-bc2e-1f636c07fa4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:32.691468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63b648-4b48-4a10-bc2e-1f636c07fa4b service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:32.701601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 896/1799] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:32.702944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4f524-ecf7-4b86-abf0-20b581ccf19c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:32.703231 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4f524-ecf7-4b86-abf0-20b581ccf19c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:32.703505 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4f524-ecf7-4b86-abf0-20b581ccf19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.714087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-942dead5-f61f-47af-b53f-60e4ae64feb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ba30ef784741fab2f65f3dc139e94e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.715052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-942dead5-f61f-47af-b53f-60e4ae64feb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ba30ef784741fab2f65f3dc139e94e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.723587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-ccf4f524-ecf7-4b86-abf0-20b581ccf19c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:32.724884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ccf4f524-ecf7-4b86-abf0-20b581ccf19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:32.729758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-942dead5-f61f-47af-b53f-60e4ae64feb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ba30ef784741fab2f65f3dc139e94e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:32.734170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 897/1800] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:32.734963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4f524-ecf7-4b86-abf0-20b581ccf19c None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:32.736786 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 904/1801] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:32 2022] DELETE /identity/v3/users/30ba30ef784741fab2f65f3dc139e94e => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:32.736933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ccf4f524-ecf7-4b86-abf0-20b581ccf19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:32.741788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384c6ef-3c2b-42c6-9e73-3cc30f834c7b None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:32.742287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384c6ef-3c2b-42c6-9e73-3cc30f834c7b None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:32.743241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384c6ef-3c2b-42c6-9e73-3cc30f834c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.751202 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4f524-ecf7-4b86-abf0-20b581ccf19c None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:32.766215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-a384c6ef-3c2b-42c6-9e73-3cc30f834c7b None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:32.767691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a384c6ef-3c2b-42c6-9e73-3cc30f834c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:32.816781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33e40d-9d95-4685-9197-a37dcd6bbb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:32.817513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33e40d-9d95-4685-9197-a37dcd6bbb50 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:32.817863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33e40d-9d95-4685-9197-a37dcd6bbb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:32.818210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33e40d-9d95-4685-9197-a37dcd6bbb50 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.818902 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33e40d-9d95-4685-9197-a37dcd6bbb50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:32.820935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33e40d-9d95-4685-9197-a37dcd6bbb50 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:32.830322 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 905/1802] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:32.832488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384c6ef-3c2b-42c6-9e73-3cc30f834c7b None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:32.841717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384c6ef-3c2b-42c6-9e73-3cc30f834c7b None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:32.846477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e83d9d-47e9-4a00-b5fe-42ab085241bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:32.847072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e83d9d-47e9-4a00-b5fe-42ab085241bf admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:32.847350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e83d9d-47e9-4a00-b5fe-42ab085241bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:32.847630 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e83d9d-47e9-4a00-b5fe-42ab085241bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.848304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e83d9d-47e9-4a00-b5fe-42ab085241bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:32.848614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e83d9d-47e9-4a00-b5fe-42ab085241bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:32.850202 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e83d9d-47e9-4a00-b5fe-42ab085241bf admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:32.861777 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 906/1803] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:32.862288 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e83d9d-47e9-4a00-b5fe-42ab085241bf admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:32.863924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-07e83d9d-47e9-4a00-b5fe-42ab085241bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:32.909734 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 898/1804] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:32 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:32.911033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33e40d-9d95-4685-9197-a37dcd6bbb50 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:32.926494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33e40d-9d95-4685-9197-a37dcd6bbb50 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:32.931682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd4ecf-8bb9-4899-b334-582b69c11daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:32.933249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd4ecf-8bb9-4899-b334-582b69c11daf admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:32.933688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd4ecf-8bb9-4899-b334-582b69c11daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:32.934097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd4ecf-8bb9-4899-b334-582b69c11daf admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.935013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd4ecf-8bb9-4899-b334-582b69c11daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:32.937110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd4ecf-8bb9-4899-b334-582b69c11daf admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:32.944587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e83d9d-47e9-4a00-b5fe-42ab085241bf admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:32.950786 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f78f9-a27d-4220-b1f5-b65f25bd2205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:32.951411 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f78f9-a27d-4220-b1f5-b65f25bd2205 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:32.951841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f78f9-a27d-4220-b1f5-b65f25bd2205 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:32.952127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f78f9-a27d-4220-b1f5-b65f25bd2205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.971742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f78f9-a27d-4220-b1f5-b65f25bd2205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:32.973416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f78f9-a27d-4220-b1f5-b65f25bd2205 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:32.984647 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 907/1805] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:32.985545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f78f9-a27d-4220-b1f5-b65f25bd2205 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:32.996735 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f78f9-a27d-4220-b1f5-b65f25bd2205 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:33.001283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec6862-846c-450c-9889-ceefda8e1afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:33.002139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec6862-846c-450c-9889-ceefda8e1afc admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:33.002326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec6862-846c-450c-9889-ceefda8e1afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:33.002720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec6862-846c-450c-9889-ceefda8e1afc admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:33.003438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec6862-846c-450c-9889-ceefda8e1afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:33.005188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec6862-846c-450c-9889-ceefda8e1afc admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:33.021953 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 908/1806] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:33.022948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec6862-846c-450c-9889-ceefda8e1afc admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:33.034075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec6862-846c-450c-9889-ceefda8e1afc admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:33.039309 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ead0f-b367-4c07-85d2-aa7432ad5dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:33.040193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ead0f-b367-4c07-85d2-aa7432ad5dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:33.040815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ead0f-b367-4c07-85d2-aa7432ad5dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:33.041025 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ead0f-b367-4c07-85d2-aa7432ad5dc6 admin admin] PATH_INFO: `/v3/projects/fc666237638643e2bad3a45c1a900f22/users/8c0a3f13ff774c64b7be696ae90bb1ca/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:33.043700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 899/1807] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:32 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:33.047690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd4ecf-8bb9-4899-b334-582b69c11daf admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:33.059225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ead0f-b367-4c07-85d2-aa7432ad5dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc666237638643e2bad3a45c1a900f22, user_id=8c0a3f13ff774c64b7be696ae90bb1ca, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:33.062091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ead0f-b367-4c07-85d2-aa7432ad5dc6 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:33.128879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd4ecf-8bb9-4899-b334-582b69c11daf admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:33.133750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58dee5-c80f-42f4-ac24-77d90126e50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:33.134175 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58dee5-c80f-42f4-ac24-77d90126e50d admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:33.134481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58dee5-c80f-42f4-ac24-77d90126e50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:33.134718 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58dee5-c80f-42f4-ac24-77d90126e50d admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:33.135954 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58dee5-c80f-42f4-ac24-77d90126e50d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:33.138174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58dee5-c80f-42f4-ac24-77d90126e50d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:33.230391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 909/1808] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:33 2022] PUT /identity/v3/projects/fc666237638643e2bad3a45c1a900f22/users/8c0a3f13ff774c64b7be696ae90bb1ca/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:33.237931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ead0f-b367-4c07-85d2-aa7432ad5dc6 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:33.262930 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 900/1809] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:33 2022] POST /identity/v3/users => generated 285 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:27:33.268657 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58dee5-c80f-42f4-ac24-77d90126e50d admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:33.281243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58dee5-c80f-42f4-ac24-77d90126e50d admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:33.377731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ead0f-b367-4c07-85d2-aa7432ad5dc6 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:33.386422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb564-2339-4ef8-892f-0cd7b553616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:33.388251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb564-2339-4ef8-892f-0cd7b553616e admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:33.388877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb564-2339-4ef8-892f-0cd7b553616e admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:33.389335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb564-2339-4ef8-892f-0cd7b553616e admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:33.390289 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedb564-2339-4ef8-892f-0cd7b553616e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:33.390411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd8f66-6958-499a-a257-72529902ee82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:33.391025 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd8f66-6958-499a-a257-72529902ee82 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:33.391505 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd8f66-6958-499a-a257-72529902ee82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:33.391819 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd8f66-6958-499a-a257-72529902ee82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:33.392847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd8f66-6958-499a-a257-72529902ee82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:33.393351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedb564-2339-4ef8-892f-0cd7b553616e admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:33.394476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd8f66-6958-499a-a257-72529902ee82 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:33.407646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 901/1811] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:33.407837 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 910/1811] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:33.409752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:33.411206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd8f66-6958-499a-a257-72529902ee82 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:33.411318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb564-2339-4ef8-892f-0cd7b553616e admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:33.428834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb564-2339-4ef8-892f-0cd7b553616e admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:33.429049 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd8f66-6958-499a-a257-72529902ee82 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:33.434800 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3ff69-e96a-4be5-b81c-eb8c777f2507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:33.440122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3ff69-e96a-4be5-b81c-eb8c777f2507 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:33.440450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1f2be-6e30-4a34-a8d8-49b5b57b503a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:33.440554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3ff69-e96a-4be5-b81c-eb8c777f2507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:33.440554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3ff69-e96a-4be5-b81c-eb8c777f2507 admin admin] PATH_INFO: `/v3/projects/fc666237638643e2bad3a45c1a900f22/users/8c0a3f13ff774c64b7be696ae90bb1ca/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:33.440697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1f2be-6e30-4a34-a8d8-49b5b57b503a admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:33.440697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1f2be-6e30-4a34-a8d8-49b5b57b503a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:33.440697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1f2be-6e30-4a34-a8d8-49b5b57b503a admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:33.440697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1f2be-6e30-4a34-a8d8-49b5b57b503a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:33.440941 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3ff69-e96a-4be5-b81c-eb8c777f2507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc666237638643e2bad3a45c1a900f22, user_id=8c0a3f13ff774c64b7be696ae90bb1ca, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:33.441037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1f2be-6e30-4a34-a8d8-49b5b57b503a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:33.441111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3ff69-e96a-4be5-b81c-eb8c777f2507 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:33.456364 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 911/1812] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:33.457325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1f2be-6e30-4a34-a8d8-49b5b57b503a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:33.460245 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 902/1813] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:33 2022] PUT /identity/v3/projects/fc666237638643e2bad3a45c1a900f22/users/8c0a3f13ff774c64b7be696ae90bb1ca/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:33.468903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea55d8-8da2-47b9-bb4a-39743562ee9a None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:33.470379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea55d8-8da2-47b9-bb4a-39743562ee9a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:33.470668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea55d8-8da2-47b9-bb4a-39743562ee9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:33.477714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1f2be-6e30-4a34-a8d8-49b5b57b503a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:33.521436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-9aea55d8-8da2-47b9-bb4a-39743562ee9a None None] MFA Rules not processed for user `8c0a3f13ff774c64b7be696ae90bb1ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:33.524151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9aea55d8-8da2-47b9-bb4a-39743562ee9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:33.546097 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a30548-d8c4-42e2-98c4-ffa5acdf95ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:33.546933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a30548-d8c4-42e2-98c4-ffa5acdf95ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:33.547217 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a30548-d8c4-42e2-98c4-ffa5acdf95ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:33.547536 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a30548-d8c4-42e2-98c4-ffa5acdf95ac admin admin] PATH_INFO: `/v3/projects/45feb051b10140e396339f89ffa2cb36/users/bf0ed986e41b4f60ae9234c545c24880/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:33.572932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a30548-d8c4-42e2-98c4-ffa5acdf95ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=45feb051b10140e396339f89ffa2cb36, user_id=bf0ed986e41b4f60ae9234c545c24880, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:33.576361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a30548-d8c4-42e2-98c4-ffa5acdf95ac admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:33.606234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 903/1814] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:33.623126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea55d8-8da2-47b9-bb4a-39743562ee9a None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:33.623126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9aea55d8-8da2-47b9-bb4a-39743562ee9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:33.698045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea55d8-8da2-47b9-bb4a-39743562ee9a None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:33.710585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fc5aa-1b7f-41f4-acfb-a0e73d868e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:33.710585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fc5aa-1b7f-41f4-acfb-a0e73d868e30 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:33.710585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fc5aa-1b7f-41f4-acfb-a0e73d868e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:33.710585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fc5aa-1b7f-41f4-acfb-a0e73d868e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:33.725096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 912/1815] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:33 2022] PUT /identity/v3/projects/45feb051b10140e396339f89ffa2cb36/users/bf0ed986e41b4f60ae9234c545c24880/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:33.727163 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a30548-d8c4-42e2-98c4-ffa5acdf95ac admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:33.735926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-f2a30548-d8c4-42e2-98c4-ffa5acdf95ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:33.737430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fc5aa-1b7f-41f4-acfb-a0e73d868e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:33.738972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fc5aa-1b7f-41f4-acfb-a0e73d868e30 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:33.763118 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 904/1816] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:33.766890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6960e0-32fe-4027-8a3b-07d9a9a78d76 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:33.768276 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6960e0-32fe-4027-8a3b-07d9a9a78d76 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:33.768709 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6960e0-32fe-4027-8a3b-07d9a9a78d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:33.813402 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-1e6960e0-32fe-4027-8a3b-07d9a9a78d76 None None] MFA Rules not processed for user `bf0ed986e41b4f60ae9234c545c24880`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:33.820782 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1e6960e0-32fe-4027-8a3b-07d9a9a78d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:33.846176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a30548-d8c4-42e2-98c4-ffa5acdf95ac admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:33.852476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:33.853106 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:33.858236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:33.858236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] PATH_INFO: `/v3/projects/f75052ceff574c0292bfebd9b0b5021b` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:33.858236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f75052ceff574c0292bfebd9b0b5021b)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:33.858236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:33.913394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 905/1817] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:33.915580 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6960e0-32fe-4027-8a3b-07d9a9a78d76 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:33.929672 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6960e0-32fe-4027-8a3b-07d9a9a78d76 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:33.936323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784bd0e-5240-4749-94b8-c79771a5adb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:33.937152 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784bd0e-5240-4749-94b8-c79771a5adb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:33.937631 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784bd0e-5240-4749-94b8-c79771a5adb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:33.938098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784bd0e-5240-4749-94b8-c79771a5adb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:33.938923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784bd0e-5240-4749-94b8-c79771a5adb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:33.942337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784bd0e-5240-4749-94b8-c79771a5adb6 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:34.035752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f75052ceff574c0292bfebd9b0b5021b', 'request_id': 'req-ae7c389f-6cea-4d42-ab04-36c544d2638f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.042497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f75052ceff574c0292bfebd9b0b5021b', 'request_id': 'req-ae7c389f-6cea-4d42-ab04-36c544d2638f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.048367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f75052ceff574c0292bfebd9b0b5021b', 'request_id': 'req-ae7c389f-6cea-4d42-ab04-36c544d2638f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.168244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] The token cache is being invalidate because project f75052ceff574c0292bfebd9b0b5021b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:34.168244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.169554 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.171169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 913/1818] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:33 2022] DELETE /identity/v3/projects/f75052ceff574c0292bfebd9b0b5021b => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:34.174758 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 906/1819] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:33 2022] POST /identity/v3/projects => generated 367 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:27:34.176610 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:34.178498 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784bd0e-5240-4749-94b8-c79771a5adb6 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:34.180832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b784bd0e-5240-4749-94b8-c79771a5adb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:34.185191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:34.296566 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784bd0e-5240-4749-94b8-c79771a5adb6 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:34.305693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c389f-6cea-4d42-ab04-36c544d2638f admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:34.307337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9a3e3-849c-4256-b0ac-0717ae9bb376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:34.309423 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a3e3-849c-4256-b0ac-0717ae9bb376 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:34.309423 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a3e3-849c-4256-b0ac-0717ae9bb376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:34.309423 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a3e3-849c-4256-b0ac-0717ae9bb376 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:34.313013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9a3e3-849c-4256-b0ac-0717ae9bb376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:34.313013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9a3e3-849c-4256-b0ac-0717ae9bb376 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:34.319361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:34.320017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:34.320334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:34.320615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] PATH_INFO: `/v3/users/feba9584d03c46e2937cacdf57e2d8d4` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:34.324728 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feba9584d03c46e2937cacdf57e2d8d4)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:34.329834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:34.372450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:34.438390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 907/1820] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:34 2022] POST /identity/v3/users => generated 290 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:34.445848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9a3e3-849c-4256-b0ac-0717ae9bb376 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:34.496561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9a3e3-849c-4256-b0ac-0717ae9bb376 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:34.504258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e286819-29d1-4b94-82e9-0a7b616d374a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:34.504970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e286819-29d1-4b94-82e9-0a7b616d374a admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:34.505863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e286819-29d1-4b94-82e9-0a7b616d374a admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:34.505863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e286819-29d1-4b94-82e9-0a7b616d374a admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:34.506322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e286819-29d1-4b94-82e9-0a7b616d374a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:34.512392 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e286819-29d1-4b94-82e9-0a7b616d374a admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:34.532462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 908/1821] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:34.533895 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e286819-29d1-4b94-82e9-0a7b616d374a admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:34.536339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9e286819-29d1-4b94-82e9-0a7b616d374a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:34.544570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:34.562463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feba9584d03c46e2937cacdf57e2d8d4'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.567207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e286819-29d1-4b94-82e9-0a7b616d374a admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:34.588126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:34.593656 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feba9584d03c46e2937cacdf57e2d8d4'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.594091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:34.594091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:34.594091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] PATH_INFO: `/v3/projects/b11ae826af3546658931245e764d270b` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:34.594091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b11ae826af3546658931245e764d270b)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:34.595992 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:34.663675 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feba9584d03c46e2937cacdf57e2d8d4'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.664142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feba9584d03c46e2937cacdf57e2d8d4'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.677688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feba9584d03c46e2937cacdf57e2d8d4'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.686721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 914/1822] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:34 2022] DELETE /identity/v3/users/feba9584d03c46e2937cacdf57e2d8d4 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:34.686721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:34.686721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:34.754018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b11ae826af3546658931245e764d270b', 'request_id': 'req-35d6d7df-2c07-44dc-a67d-5279a786da41'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.760290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b11ae826af3546658931245e764d270b', 'request_id': 'req-35d6d7df-2c07-44dc-a67d-5279a786da41'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.778967 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05ad6a-4fa2-4743-96ea-ca11266a5668 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:34.836884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344b570-86b1-4d4c-9471-22d8214a926d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:34.837545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344b570-86b1-4d4c-9471-22d8214a926d admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:34.837829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344b570-86b1-4d4c-9471-22d8214a926d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:34.838099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344b570-86b1-4d4c-9471-22d8214a926d admin admin] PATH_INFO: `/v3/projects/cfd66a74ff6b4958a9e9d5751c6743d4/users/7d253c5f6e8f4af980ba547a445e7028/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:34.840271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b11ae826af3546658931245e764d270b', 'request_id': 'req-35d6d7df-2c07-44dc-a67d-5279a786da41'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.854420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344b570-86b1-4d4c-9471-22d8214a926d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfd66a74ff6b4958a9e9d5751c6743d4, user_id=7d253c5f6e8f4af980ba547a445e7028, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:34.854597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] The token cache is being invalidate because project b11ae826af3546658931245e764d270b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:34.854851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.855570 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:34.859878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344b570-86b1-4d4c-9471-22d8214a926d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:34.860271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 909/1823] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:34 2022] DELETE /identity/v3/projects/b11ae826af3546658931245e764d270b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:34.861475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:34.862870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:34.876489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6d7df-2c07-44dc-a67d-5279a786da41 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:34.882121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:34.883281 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:34.883281 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:34.891889 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] PATH_INFO: `/v3/users/2b3580a62306434bbfeee6cf9ccac53e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:34.891889 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b3580a62306434bbfeee6cf9ccac53e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:34.891889 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:34.944327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 915/1824] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:34 2022] PUT /identity/v3/projects/cfd66a74ff6b4958a9e9d5751c6743d4/users/7d253c5f6e8f4af980ba547a445e7028/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:34.952404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344b570-86b1-4d4c-9471-22d8214a926d admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:34.952790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3344b570-86b1-4d4c-9471-22d8214a926d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:35.020335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344b570-86b1-4d4c-9471-22d8214a926d admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:35.026863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405509a-e76e-4eab-9763-c8c28233fb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:35.027476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405509a-e76e-4eab-9763-c8c28233fb1c admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:35.027787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405509a-e76e-4eab-9763-c8c28233fb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:35.028271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405509a-e76e-4eab-9763-c8c28233fb1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:35.028837 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405509a-e76e-4eab-9763-c8c28233fb1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:35.031014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405509a-e76e-4eab-9763-c8c28233fb1c admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:35.047059 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 916/1825] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:35.048115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405509a-e76e-4eab-9763-c8c28233fb1c admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:35.059035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405509a-e76e-4eab-9763-c8c28233fb1c admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:35.063889 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9ffbc-c69c-4a30-98b9-46d794e62c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:35.064579 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9ffbc-c69c-4a30-98b9-46d794e62c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:35.068636 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9ffbc-c69c-4a30-98b9-46d794e62c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:35.068636 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9ffbc-c69c-4a30-98b9-46d794e62c45 admin admin] PATH_INFO: `/v3/projects/cfd66a74ff6b4958a9e9d5751c6743d4/users/7d253c5f6e8f4af980ba547a445e7028/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:35.068636 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9ffbc-c69c-4a30-98b9-46d794e62c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfd66a74ff6b4958a9e9d5751c6743d4, user_id=7d253c5f6e8f4af980ba547a445e7028, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:35.070079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9ffbc-c69c-4a30-98b9-46d794e62c45 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:35.098244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 917/1826] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:35 2022] PUT /identity/v3/projects/cfd66a74ff6b4958a9e9d5751c6743d4/users/7d253c5f6e8f4af980ba547a445e7028/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:35.110345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2030b9-4a1b-4765-afa1-daf279950fd5 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:35.110742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2030b9-4a1b-4765-afa1-daf279950fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:35.111022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2030b9-4a1b-4765-afa1-daf279950fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:35.141666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ca2030b9-4a1b-4765-afa1-daf279950fd5 None None] MFA Rules not processed for user `7d253c5f6e8f4af980ba547a445e7028`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:35.143151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ca2030b9-4a1b-4765-afa1-daf279950fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:35.216842 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 918/1827] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:35.221194 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2030b9-4a1b-4765-afa1-daf279950fd5 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:35.222781 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ca2030b9-4a1b-4765-afa1-daf279950fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:35.241911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b3580a62306434bbfeee6cf9ccac53e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.242568 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b3580a62306434bbfeee6cf9ccac53e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.250014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b3580a62306434bbfeee6cf9ccac53e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.302204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2030b9-4a1b-4765-afa1-daf279950fd5 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:35.307298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:35.308580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:35.308859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:35.309130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] PATH_INFO: `/v3/users/6263de5280c64c5bb1637ca7b95ea501` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:35.310271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6263de5280c64c5bb1637ca7b95ea501)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:35.312279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:35.366035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b3580a62306434bbfeee6cf9ccac53e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.366876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b3580a62306434bbfeee6cf9ccac53e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.389648 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 910/1828] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:34 2022] DELETE /identity/v3/users/2b3580a62306434bbfeee6cf9ccac53e => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:35.390432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:35.394245 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:35.479522 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f1154-bb71-4c44-9705-6deaec0e7593 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:35.485010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fc3ee-934a-4ae4-9423-9a5da623a6bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:35.485693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fc3ee-934a-4ae4-9423-9a5da623a6bf service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:35.485982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fc3ee-934a-4ae4-9423-9a5da623a6bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:35.489749 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fc3ee-934a-4ae4-9423-9a5da623a6bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:35.489749 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-033fc3ee-934a-4ae4-9423-9a5da623a6bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:35.562498 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033fc3ee-934a-4ae4-9423-9a5da623a6bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:35.564478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033fc3ee-934a-4ae4-9423-9a5da623a6bf service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:35.564650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6263de5280c64c5bb1637ca7b95ea501'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.575788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6263de5280c64c5bb1637ca7b95ea501'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.579271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 911/1829] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:35.580541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fc3ee-934a-4ae4-9423-9a5da623a6bf service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:35.583099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-033fc3ee-934a-4ae4-9423-9a5da623a6bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:35.695879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fc3ee-934a-4ae4-9423-9a5da623a6bf service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:35.715724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:35.715724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:35.715724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:35.715724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9 admin admin] PATH_INFO: `/v3/projects/a3cccc301a2f4d47a1df698442b7b240` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:35.716111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3cccc301a2f4d47a1df698442b7b240)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:35.716111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:35.733947 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6263de5280c64c5bb1637ca7b95ea501'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.736570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6263de5280c64c5bb1637ca7b95ea501'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.764989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6263de5280c64c5bb1637ca7b95ea501'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.780351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 919/1830] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:35 2022] DELETE /identity/v3/users/6263de5280c64c5bb1637ca7b95ea501 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:35.781531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:35.784387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:35.849329 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3cccc301a2f4d47a1df698442b7b240', 'request_id': 'req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.855984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3cccc301a2f4d47a1df698442b7b240', 'request_id': 'req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.898894 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3b243-39b3-4112-bdb5-e2ef5830da75 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:35.925703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3cccc301a2f4d47a1df698442b7b240', 'request_id': 'req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.939755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9 admin admin] The token cache is being invalidate because project a3cccc301a2f4d47a1df698442b7b240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:35.940377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.940902 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d1a8ecfc-0778-4823-b6e2-4624e295e3c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:35.945894 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 912/1831] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:35 2022] DELETE /identity/v3/projects/a3cccc301a2f4d47a1df698442b7b240 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:35.945894 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbc4bf-6b72-444b-a748-bd2165181140 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:35.945894 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbc4bf-6b72-444b-a748-bd2165181140 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:35.945894 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbc4bf-6b72-444b-a748-bd2165181140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:35.965801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8d244-61e7-4667-8006-9c2d876b0b4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:35.966864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8d244-61e7-4667-8006-9c2d876b0b4a service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:35.967287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8d244-61e7-4667-8006-9c2d876b0b4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:35.967519 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-33fbc4bf-6b72-444b-a748-bd2165181140 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:35.967879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8d244-61e7-4667-8006-9c2d876b0b4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:35.968781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-33fbc4bf-6b72-444b-a748-bd2165181140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:35.970251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2df8d244-61e7-4667-8006-9c2d876b0b4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:36.033829 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 913/1832] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:36.035237 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbc4bf-6b72-444b-a748-bd2165181140 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:36.036595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-33fbc4bf-6b72-444b-a748-bd2165181140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:36.038314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8d244-61e7-4667-8006-9c2d876b0b4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:36.040039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8d244-61e7-4667-8006-9c2d876b0b4a service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:36.050413 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 920/1833] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:36.056217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbc4bf-6b72-444b-a748-bd2165181140 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:36.061342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:36.061932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f3f07-af4a-4954-9dbc-8ac7e4eee288 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.062134 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.062289 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f3f07-af4a-4954-9dbc-8ac7e4eee288 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.062435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.062686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f3f07-af4a-4954-9dbc-8ac7e4eee288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.062784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] PATH_INFO: `/v3/projects/cb4e8d42e3134e97a9ed23f99cadf104` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.063803 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb4e8d42e3134e97a9ed23f99cadf104)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:36.065750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:36.083856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-2d8f3f07-af4a-4954-9dbc-8ac7e4eee288 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:36.084945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2d8f3f07-af4a-4954-9dbc-8ac7e4eee288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:36.093986 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 921/1834] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:36.094807 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f3f07-af4a-4954-9dbc-8ac7e4eee288 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:36.096102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2d8f3f07-af4a-4954-9dbc-8ac7e4eee288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:36.159139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f3f07-af4a-4954-9dbc-8ac7e4eee288 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:36.163246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c7f7d-777a-4609-b37a-29998a384e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:36.163863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c7f7d-777a-4609-b37a-29998a384e57 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.164147 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c7f7d-777a-4609-b37a-29998a384e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.164804 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c7f7d-777a-4609-b37a-29998a384e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.177075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c7f7d-777a-4609-b37a-29998a384e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:36.178656 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c7f7d-777a-4609-b37a-29998a384e57 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:36.191020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 922/1835] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:36.192997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1cb8b-7c8a-4601-9f17-012356229956 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.193286 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1cb8b-7c8a-4601-9f17-012356229956 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.193567 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1cb8b-7c8a-4601-9f17-012356229956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.216700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e6f1cb8b-7c8a-4601-9f17-012356229956 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:36.217925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e6f1cb8b-7c8a-4601-9f17-012356229956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:36.235251 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb4e8d42e3134e97a9ed23f99cadf104', 'request_id': 'req-0fff4d06-0822-46de-b4d9-527595897a3f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:36.243177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb4e8d42e3134e97a9ed23f99cadf104', 'request_id': 'req-0fff4d06-0822-46de-b4d9-527595897a3f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:36.293432 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 923/1836] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:36.296296 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1cb8b-7c8a-4601-9f17-012356229956 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:36.306225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1cb8b-7c8a-4601-9f17-012356229956 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:36.319289 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba051743-b054-4f23-af04-e71c6b569efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:36.320153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba051743-b054-4f23-af04-e71c6b569efd admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.320153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba051743-b054-4f23-af04-e71c6b569efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.320349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba051743-b054-4f23-af04-e71c6b569efd admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.321192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba051743-b054-4f23-af04-e71c6b569efd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:36.321700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba051743-b054-4f23-af04-e71c6b569efd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:36.323279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba051743-b054-4f23-af04-e71c6b569efd admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:36.326555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb4e8d42e3134e97a9ed23f99cadf104', 'request_id': 'req-0fff4d06-0822-46de-b4d9-527595897a3f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:36.345724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] The token cache is being invalidate because project cb4e8d42e3134e97a9ed23f99cadf104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:36.346424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:36.347479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:36.348851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 924/1837] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:36.350186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 914/1838] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:36 2022] DELETE /identity/v3/projects/cb4e8d42e3134e97a9ed23f99cadf104 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:36.360680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:36.361595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc94cd3-3345-4161-882b-4a3595506fc1 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.361883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc94cd3-3345-4161-882b-4a3595506fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.367384 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc94cd3-3345-4161-882b-4a3595506fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.367512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:36.390806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ccc94cd3-3345-4161-882b-4a3595506fc1 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:36.392470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ccc94cd3-3345-4161-882b-4a3595506fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:36.396828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff4d06-0822-46de-b4d9-527595897a3f admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:36.402649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae20145-09df-4405-97a9-3825c7caf9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:36.403255 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae20145-09df-4405-97a9-3825c7caf9db admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.403534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae20145-09df-4405-97a9-3825c7caf9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.403821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae20145-09df-4405-97a9-3825c7caf9db admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.404873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 925/1839] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:36.410730 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae20145-09df-4405-97a9-3825c7caf9db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:36.412046 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f3a39-c301-45c4-9ba9-ab757af1e205 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.412427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f3a39-c301-45c4-9ba9-ab757af1e205 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.412710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f3a39-c301-45c4-9ba9-ab757af1e205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.413031 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae20145-09df-4405-97a9-3825c7caf9db admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:36.438772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-362f3a39-c301-45c4-9ba9-ab757af1e205 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:36.441631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-362f3a39-c301-45c4-9ba9-ab757af1e205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:36.453438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 926/1840] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:36.458694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f3a39-c301-45c4-9ba9-ab757af1e205 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:36.468301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f3a39-c301-45c4-9ba9-ab757af1e205 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:36.482084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a7806-1db8-4700-9daa-936b1393da45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:36.482729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a7806-1db8-4700-9daa-936b1393da45 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.483010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a7806-1db8-4700-9daa-936b1393da45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.483286 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a7806-1db8-4700-9daa-936b1393da45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.493316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a7806-1db8-4700-9daa-936b1393da45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:36.493316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a7806-1db8-4700-9daa-936b1393da45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:36.494580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a7806-1db8-4700-9daa-936b1393da45 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:36.516021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 927/1841] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:36.517094 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a7806-1db8-4700-9daa-936b1393da45 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:36.529985 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a7806-1db8-4700-9daa-936b1393da45 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:36.538124 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eac78-7251-415b-bcfb-307528960cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:36.538474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eac78-7251-415b-bcfb-307528960cfb admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.538760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eac78-7251-415b-bcfb-307528960cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.539031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eac78-7251-415b-bcfb-307528960cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.540272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907eac78-7251-415b-bcfb-307528960cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:36.545017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907eac78-7251-415b-bcfb-307528960cfb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:36.574556 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 915/1842] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:36 2022] POST /identity/v3/projects => generated 387 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:36.574990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae20145-09df-4405-97a9-3825c7caf9db admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:36.672603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae20145-09df-4405-97a9-3825c7caf9db admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:36.677634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4f301-7571-421c-a390-020049b1f9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:36.678419 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4f301-7571-421c-a390-020049b1f9fa admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.678852 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4f301-7571-421c-a390-020049b1f9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.679255 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4f301-7571-421c-a390-020049b1f9fa admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.680098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4f301-7571-421c-a390-020049b1f9fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:36.682219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4f301-7571-421c-a390-020049b1f9fa admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:36.766894 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 928/1843] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:36 2022] POST /identity/v3/projects => generated 379 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:36.771141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eac78-7251-415b-bcfb-307528960cfb admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:36.862806 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 916/1844] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:36 2022] POST /identity/v3/users => generated 294 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:36.874469 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eac78-7251-415b-bcfb-307528960cfb admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:36.875350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4f301-7571-421c-a390-020049b1f9fa admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:36.881121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800db32a-3dff-453d-868d-b4f418441cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:36.881121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800db32a-3dff-453d-868d-b4f418441cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.881461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800db32a-3dff-453d-868d-b4f418441cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.882214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800db32a-3dff-453d-868d-b4f418441cc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.883025 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800db32a-3dff-453d-868d-b4f418441cc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:36.885030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800db32a-3dff-453d-868d-b4f418441cc6 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:36.890130 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4f301-7571-421c-a390-020049b1f9fa admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:36.899743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0883a-6c7c-403e-b094-48fe432ae99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:36.900485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0883a-6c7c-403e-b094-48fe432ae99e admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.900859 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0883a-6c7c-403e-b094-48fe432ae99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.901409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0883a-6c7c-403e-b094-48fe432ae99e admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.902070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0883a-6c7c-403e-b094-48fe432ae99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:36.907424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0883a-6c7c-403e-b094-48fe432ae99e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:36.944091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 917/1845] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:27:36.944091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0883a-6c7c-403e-b094-48fe432ae99e admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:36.944091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ffb0883a-6c7c-403e-b094-48fe432ae99e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:36.956821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0883a-6c7c-403e-b094-48fe432ae99e admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:36.963448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b313b2f-d846-4335-bb31-dec10f7ac712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:36.964125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313b2f-d846-4335-bb31-dec10f7ac712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:36.964487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313b2f-d846-4335-bb31-dec10f7ac712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:36.964696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313b2f-d846-4335-bb31-dec10f7ac712 admin admin] PATH_INFO: `/v3/projects/504ac0aeb5c64252b3c39fb31254cffc` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:36.966031 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b313b2f-d846-4335-bb31-dec10f7ac712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=504ac0aeb5c64252b3c39fb31254cffc)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:36.967946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b313b2f-d846-4335-bb31-dec10f7ac712 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:37.004013 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 929/1846] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:36 2022] POST /identity/v3/users => generated 290 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:27:37.004013 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800db32a-3dff-453d-868d-b4f418441cc6 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:37.013541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800db32a-3dff-453d-868d-b4f418441cc6 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:37.019665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5866eda-7669-4bc5-a942-d8ded0b55d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:37.020619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5866eda-7669-4bc5-a942-d8ded0b55d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.020907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5866eda-7669-4bc5-a942-d8ded0b55d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.021243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5866eda-7669-4bc5-a942-d8ded0b55d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.022731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5866eda-7669-4bc5-a942-d8ded0b55d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:37.024424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5866eda-7669-4bc5-a942-d8ded0b55d81 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:37.034923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 930/1847] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:37.036553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5866eda-7669-4bc5-a942-d8ded0b55d81 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:37.049440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5866eda-7669-4bc5-a942-d8ded0b55d81 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:37.054278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ae697-ff26-4c32-a8bc-bf00eb675a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:37.056861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ae697-ff26-4c32-a8bc-bf00eb675a86 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.056861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ae697-ff26-4c32-a8bc-bf00eb675a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.056861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ae697-ff26-4c32-a8bc-bf00eb675a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.056861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ae697-ff26-4c32-a8bc-bf00eb675a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:37.058269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ae697-ff26-4c32-a8bc-bf00eb675a86 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:37.076544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 931/1848] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:37.077449 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ae697-ff26-4c32-a8bc-bf00eb675a86 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:37.090322 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ae697-ff26-4c32-a8bc-bf00eb675a86 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:37.102241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661069d-447f-477a-9234-22eda0b23ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:37.102975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661069d-447f-477a-9234-22eda0b23ded admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.103319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661069d-447f-477a-9234-22eda0b23ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.103688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661069d-447f-477a-9234-22eda0b23ded admin admin] PATH_INFO: `/v3/projects/c9fee0114516417f982510947e169cb2/users/85b7791eb6164be59465e506e4edf559/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.125743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d661069d-447f-477a-9234-22eda0b23ded admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9fee0114516417f982510947e169cb2, user_id=85b7791eb6164be59465e506e4edf559, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:37.129002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d661069d-447f-477a-9234-22eda0b23ded admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:37.139500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b313b2f-d846-4335-bb31-dec10f7ac712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '504ac0aeb5c64252b3c39fb31254cffc', 'request_id': 'req-2b313b2f-d846-4335-bb31-dec10f7ac712'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:37.145298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b313b2f-d846-4335-bb31-dec10f7ac712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '504ac0aeb5c64252b3c39fb31254cffc', 'request_id': 'req-2b313b2f-d846-4335-bb31-dec10f7ac712'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:37.246871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b313b2f-d846-4335-bb31-dec10f7ac712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '504ac0aeb5c64252b3c39fb31254cffc', 'request_id': 'req-2b313b2f-d846-4335-bb31-dec10f7ac712'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:37.253206 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 932/1849] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:37 2022] PUT /identity/v3/projects/c9fee0114516417f982510947e169cb2/users/85b7791eb6164be59465e506e4edf559/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:37.254268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661069d-447f-477a-9234-22eda0b23ded admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:37.270867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b313b2f-d846-4335-bb31-dec10f7ac712 admin admin] The token cache is being invalidate because project 504ac0aeb5c64252b3c39fb31254cffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:37.271211 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b313b2f-d846-4335-bb31-dec10f7ac712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:37.271918 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b313b2f-d846-4335-bb31-dec10f7ac712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:37.273767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 918/1850] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:36 2022] DELETE /identity/v3/projects/504ac0aeb5c64252b3c39fb31254cffc => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:37.275167 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34cd9f-1cfd-480e-9c6a-d200c13a9582 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.275167 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34cd9f-1cfd-480e-9c6a-d200c13a9582 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.279417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34cd9f-1cfd-480e-9c6a-d200c13a9582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.308921 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7b34cd9f-1cfd-480e-9c6a-d200c13a9582 None None] MFA Rules not processed for user `85b7791eb6164be59465e506e4edf559`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:37.310304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7b34cd9f-1cfd-480e-9c6a-d200c13a9582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:37.329507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661069d-447f-477a-9234-22eda0b23ded admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:37.334642 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33c6f8-05c8-456c-96ea-1a9dd355e2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:37.335253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33c6f8-05c8-456c-96ea-1a9dd355e2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.335531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33c6f8-05c8-456c-96ea-1a9dd355e2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.335799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33c6f8-05c8-456c-96ea-1a9dd355e2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.336516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a33c6f8-05c8-456c-96ea-1a9dd355e2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:37.338117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a33c6f8-05c8-456c-96ea-1a9dd355e2e6 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:37.348975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 933/1851] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:37.349910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33c6f8-05c8-456c-96ea-1a9dd355e2e6 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:37.351550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2a33c6f8-05c8-456c-96ea-1a9dd355e2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:37.366717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33c6f8-05c8-456c-96ea-1a9dd355e2e6 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:37.373830 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:37.383066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.383239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.383505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] PATH_INFO: `/v3/users/8c0a3f13ff774c64b7be696ae90bb1ca` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.385296 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c0a3f13ff774c64b7be696ae90bb1ca)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:37.388217 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:37.416817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 919/1852] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:37.429798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34cd9f-1cfd-480e-9c6a-d200c13a9582 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:37.429798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7b34cd9f-1cfd-480e-9c6a-d200c13a9582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:37.441109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34cd9f-1cfd-480e-9c6a-d200c13a9582 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:37.446705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814228b0-e033-4aa6-a139-e85f61fdfa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:37.447384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814228b0-e033-4aa6-a139-e85f61fdfa98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.447668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814228b0-e033-4aa6-a139-e85f61fdfa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.447948 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814228b0-e033-4aa6-a139-e85f61fdfa98 admin admin] PATH_INFO: `/v3/projects/43c3b64ca1bc4f219042011d1c8a5b52/users/bcd23774168849c4b4f1e70b42cfe6f8/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.465796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814228b0-e033-4aa6-a139-e85f61fdfa98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c3b64ca1bc4f219042011d1c8a5b52, user_id=bcd23774168849c4b4f1e70b42cfe6f8, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:37.475641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814228b0-e033-4aa6-a139-e85f61fdfa98 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:37.547762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 920/1853] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:37 2022] PUT /identity/v3/projects/43c3b64ca1bc4f219042011d1c8a5b52/users/bcd23774168849c4b4f1e70b42cfe6f8/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:37.547762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814228b0-e033-4aa6-a139-e85f61fdfa98 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:37.547762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-814228b0-e033-4aa6-a139-e85f61fdfa98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:37.623074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c0a3f13ff774c64b7be696ae90bb1ca'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:37.630893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814228b0-e033-4aa6-a139-e85f61fdfa98 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:37.631861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c0a3f13ff774c64b7be696ae90bb1ca'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:37.637454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c98f44-cee7-4afc-8d7d-de2782e772f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:37.638088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c98f44-cee7-4afc-8d7d-de2782e772f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.638368 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c98f44-cee7-4afc-8d7d-de2782e772f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.638673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c98f44-cee7-4afc-8d7d-de2782e772f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.639322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c98f44-cee7-4afc-8d7d-de2782e772f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:37.641297 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c98f44-cee7-4afc-8d7d-de2782e772f9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:37.766175 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c0a3f13ff774c64b7be696ae90bb1ca'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:37.766809 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c0a3f13ff774c64b7be696ae90bb1ca'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:37.773965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 921/1854] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:37 2022] POST /identity/v3/projects => generated 385 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:37.775324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea2279-ff88-416b-87cf-07c063621e1d None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.775619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea2279-ff88-416b-87cf-07c063621e1d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.775912 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea2279-ff88-416b-87cf-07c063621e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.782276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c0a3f13ff774c64b7be696ae90bb1ca'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:37.788595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 934/1855] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:37 2022] DELETE /identity/v3/users/8c0a3f13ff774c64b7be696ae90bb1ca => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:37.789474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:37.790827 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:37.809298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-bfea2279-ff88-416b-87cf-07c063621e1d None None] MFA Rules not processed for user `bcd23774168849c4b4f1e70b42cfe6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:37.810547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-bfea2279-ff88-416b-87cf-07c063621e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:37.889995 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee30b22-6b14-446c-853c-21f76757304a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:37.900864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 922/1856] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:37.902616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7a02e-a16e-40ea-87ce-ffe0c910089a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:37.903467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7a02e-a16e-40ea-87ce-ffe0c910089a admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.903747 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7a02e-a16e-40ea-87ce-ffe0c910089a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.904023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7a02e-a16e-40ea-87ce-ffe0c910089a admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.904749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e7a02e-a16e-40ea-87ce-ffe0c910089a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:37.912313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e7a02e-a16e-40ea-87ce-ffe0c910089a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:37.913571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea2279-ff88-416b-87cf-07c063621e1d None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:37.915055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-bfea2279-ff88-416b-87cf-07c063621e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:37.930643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea2279-ff88-416b-87cf-07c063621e1d None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:37.936114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3577725-ef28-443f-bdd0-27618cfac4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:37.936772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3577725-ef28-443f-bdd0-27618cfac4d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.937050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3577725-ef28-443f-bdd0-27618cfac4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.937322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3577725-ef28-443f-bdd0-27618cfac4d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.937963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3577725-ef28-443f-bdd0-27618cfac4d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:37.939870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3577725-ef28-443f-bdd0-27618cfac4d4 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:37.989903 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 935/1857] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:37 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:37.994042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7a02e-a16e-40ea-87ce-ffe0c910089a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:38.007718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7a02e-a16e-40ea-87ce-ffe0c910089a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:38.013833 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86592d6-4a44-4e57-88b9-3449c5dcd720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:38.014436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86592d6-4a44-4e57-88b9-3449c5dcd720 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:38.014717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86592d6-4a44-4e57-88b9-3449c5dcd720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:38.014996 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86592d6-4a44-4e57-88b9-3449c5dcd720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:38.015632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86592d6-4a44-4e57-88b9-3449c5dcd720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:38.018358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:38.024291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86592d6-4a44-4e57-88b9-3449c5dcd720 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:38.039694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 936/1858] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:27:38.044478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86592d6-4a44-4e57-88b9-3449c5dcd720 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:38.054625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86592d6-4a44-4e57-88b9-3449c5dcd720 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:38.061495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2b0d8-29ee-40dd-b95e-e65ffe6aa435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:38.062163 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2b0d8-29ee-40dd-b95e-e65ffe6aa435 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:38.062528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2b0d8-29ee-40dd-b95e-e65ffe6aa435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:38.062813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2b0d8-29ee-40dd-b95e-e65ffe6aa435 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:38.063471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2b0d8-29ee-40dd-b95e-e65ffe6aa435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:38.065115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2b0d8-29ee-40dd-b95e-e65ffe6aa435 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:38.083801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 937/1859] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:27:38.084968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2b0d8-29ee-40dd-b95e-e65ffe6aa435 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:38.090353 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 923/1860] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:37 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:38.091773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3577725-ef28-443f-bdd0-27618cfac4d4 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:38.097114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2b0d8-29ee-40dd-b95e-e65ffe6aa435 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:38.120962 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:38.170630 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c23bc-9de3-4c4e-919f-b2d8b362b670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:38.171806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c23bc-9de3-4c4e-919f-b2d8b362b670 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:38.172340 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c23bc-9de3-4c4e-919f-b2d8b362b670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:38.173412 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c23bc-9de3-4c4e-919f-b2d8b362b670 admin admin] PATH_INFO: `/v3/projects/9e1c5a8077f34ee59168a109baf082c1/users/0b661042e627417ea0910afd43ce8061/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:38.184923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3577725-ef28-443f-bdd0-27618cfac4d4 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:38.190416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c0cd7-8fa0-4296-8cd4-c21e8d307629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:38.191004 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c0cd7-8fa0-4296-8cd4-c21e8d307629 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:38.191440 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c0cd7-8fa0-4296-8cd4-c21e8d307629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:38.191719 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c0cd7-8fa0-4296-8cd4-c21e8d307629 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:38.192396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c0cd7-8fa0-4296-8cd4-c21e8d307629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:38.194449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c0cd7-8fa0-4296-8cd4-c21e8d307629 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:38.200449 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c23bc-9de3-4c4e-919f-b2d8b362b670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e1c5a8077f34ee59168a109baf082c1, user_id=0b661042e627417ea0910afd43ce8061, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:38.200449 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c23bc-9de3-4c4e-919f-b2d8b362b670 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:38.270084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 938/1861] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:38 2022] PUT /identity/v3/projects/9e1c5a8077f34ee59168a109baf082c1/users/0b661042e627417ea0910afd43ce8061/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:38.289195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc295d-a50c-404d-b7d4-27c06ef6c82b None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:38.292323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc295d-a50c-404d-b7d4-27c06ef6c82b None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:38.293001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc295d-a50c-404d-b7d4-27c06ef6c82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:38.317044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 924/1862] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:38 2022] POST /identity/v3/users => generated 290 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:38.318355 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c0cd7-8fa0-4296-8cd4-c21e8d307629 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:38.324054 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-133c0cd7-8fa0-4296-8cd4-c21e8d307629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:38.337556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-8ffc295d-a50c-404d-b7d4-27c06ef6c82b None None] MFA Rules not processed for user `0b661042e627417ea0910afd43ce8061`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:38.339341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8ffc295d-a50c-404d-b7d4-27c06ef6c82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:38.359270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c0cd7-8fa0-4296-8cd4-c21e8d307629 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:38.425142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 939/1863] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:38.425891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc295d-a50c-404d-b7d4-27c06ef6c82b None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:38.427568 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:38.428645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:38.429099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:38.429538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] PATH_INFO: `/v3/projects/fc666237638643e2bad3a45c1a900f22` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:38.431334 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc666237638643e2bad3a45c1a900f22)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:38.434760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:38.436876 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc295d-a50c-404d-b7d4-27c06ef6c82b None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:38.458171 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee765684-01f4-4f52-a5d1-94ddd752db71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:38.459700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee765684-01f4-4f52-a5d1-94ddd752db71 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:38.460861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee765684-01f4-4f52-a5d1-94ddd752db71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:38.461561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee765684-01f4-4f52-a5d1-94ddd752db71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:38.462879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee765684-01f4-4f52-a5d1-94ddd752db71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:38.466762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee765684-01f4-4f52-a5d1-94ddd752db71 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:38.479234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 940/1864] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:38.480561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee765684-01f4-4f52-a5d1-94ddd752db71 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:38.491544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee765684-01f4-4f52-a5d1-94ddd752db71 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:38.497646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7c6e6-c73a-4017-b8e3-5b58a01755af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:38.498270 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7c6e6-c73a-4017-b8e3-5b58a01755af admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:38.498554 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7c6e6-c73a-4017-b8e3-5b58a01755af admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:38.498829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7c6e6-c73a-4017-b8e3-5b58a01755af admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:38.499471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7c6e6-c73a-4017-b8e3-5b58a01755af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:38.501693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7c6e6-c73a-4017-b8e3-5b58a01755af admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:38.594739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc666237638643e2bad3a45c1a900f22', 'request_id': 'req-a9f18930-45da-421b-9504-e53744186e1e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:38.602649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc666237638643e2bad3a45c1a900f22', 'request_id': 'req-a9f18930-45da-421b-9504-e53744186e1e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:38.640285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 941/1865] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:38 2022] POST /identity/v3/projects => generated 387 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:38.640285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7c6e6-c73a-4017-b8e3-5b58a01755af admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:38.693753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc666237638643e2bad3a45c1a900f22', 'request_id': 'req-a9f18930-45da-421b-9504-e53744186e1e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:38.716877 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] The token cache is being invalidate because project fc666237638643e2bad3a45c1a900f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:38.717176 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:38.720528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:38.724711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 925/1866] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:38 2022] DELETE /identity/v3/projects/fc666237638643e2bad3a45c1a900f22 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:38.726380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:38.729427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:38.744461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7c6e6-c73a-4017-b8e3-5b58a01755af admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:38.750896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1907f9-aa4b-4a1d-82ca-0dd73c98a803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:38.751722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1907f9-aa4b-4a1d-82ca-0dd73c98a803 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:38.752272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1907f9-aa4b-4a1d-82ca-0dd73c98a803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:38.752831 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1907f9-aa4b-4a1d-82ca-0dd73c98a803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:38.753768 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1907f9-aa4b-4a1d-82ca-0dd73c98a803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:38.755565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1907f9-aa4b-4a1d-82ca-0dd73c98a803 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:38.771474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 942/1867] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:38.773244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48735e8e-b26c-41e7-9605-e41a276a73ca None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:38.773712 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48735e8e-b26c-41e7-9605-e41a276a73ca None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:38.774274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48735e8e-b26c-41e7-9605-e41a276a73ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:38.802225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-48735e8e-b26c-41e7-9605-e41a276a73ca None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:38.806295 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-48735e8e-b26c-41e7-9605-e41a276a73ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:38.818788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f18930-45da-421b-9504-e53744186e1e admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:38.825938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 943/1868] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:38.825938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48735e8e-b26c-41e7-9605-e41a276a73ca None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:38.825938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-48735e8e-b26c-41e7-9605-e41a276a73ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:38.826601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f0e64-5f48-4a43-a16b-4f8dfab1b36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:38.827144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f0e64-5f48-4a43-a16b-4f8dfab1b36e admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:38.828006 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f0e64-5f48-4a43-a16b-4f8dfab1b36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:38.832900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f0e64-5f48-4a43-a16b-4f8dfab1b36e admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:38.834190 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f0e64-5f48-4a43-a16b-4f8dfab1b36e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:38.837725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f0e64-5f48-4a43-a16b-4f8dfab1b36e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:38.840051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48735e8e-b26c-41e7-9605-e41a276a73ca None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:38.849446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4afada-ba2b-4b9b-812b-cc051634a04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:38.850353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4afada-ba2b-4b9b-812b-cc051634a04d admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:38.850676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4afada-ba2b-4b9b-812b-cc051634a04d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:38.850957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4afada-ba2b-4b9b-812b-cc051634a04d admin admin] PATH_INFO: `/v3/projects/3093f53915e34cb4a96ab1acc3fdba2b/users/b77e2f374d3c4e73897ff358681bdb74/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:38.871363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4afada-ba2b-4b9b-812b-cc051634a04d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3093f53915e34cb4a96ab1acc3fdba2b, user_id=b77e2f374d3c4e73897ff358681bdb74, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:38.871363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4afada-ba2b-4b9b-812b-cc051634a04d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:38.926767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 926/1869] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:38 2022] POST /identity/v3/users => generated 300 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:38.931500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3db30-964a-4120-8f5c-a13f215a619e None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:38.931787 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3db30-964a-4120-8f5c-a13f215a619e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:38.932070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3db30-964a-4120-8f5c-a13f215a619e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:38.950309 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 944/1870] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:38 2022] PUT /identity/v3/projects/3093f53915e34cb4a96ab1acc3fdba2b/users/b77e2f374d3c4e73897ff358681bdb74/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:38.951928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4afada-ba2b-4b9b-812b-cc051634a04d admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:38.970550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-e1d3db30-964a-4120-8f5c-a13f215a619e None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:38.972111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e1d3db30-964a-4120-8f5c-a13f215a619e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:39.049489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 927/1871] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:39.050817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa381cc-74ff-4932-b4e1-1106d905c629 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.051105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa381cc-74ff-4932-b4e1-1106d905c629 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.051380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa381cc-74ff-4932-b4e1-1106d905c629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.060660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4afada-ba2b-4b9b-812b-cc051634a04d admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.081064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-0fa381cc-74ff-4932-b4e1-1106d905c629 None None] MFA Rules not processed for user `b77e2f374d3c4e73897ff358681bdb74`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:39.082111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0832e4-f255-4fca-b3c2-b743a9a9b14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.082313 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0fa381cc-74ff-4932-b4e1-1106d905c629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:39.082768 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0832e4-f255-4fca-b3c2-b743a9a9b14d admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.083053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0832e4-f255-4fca-b3c2-b743a9a9b14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.083325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0832e4-f255-4fca-b3c2-b743a9a9b14d admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.084047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0832e4-f255-4fca-b3c2-b743a9a9b14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.085742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0832e4-f255-4fca-b3c2-b743a9a9b14d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.101261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 945/1872] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:39.101261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0832e4-f255-4fca-b3c2-b743a9a9b14d admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.111698 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0832e4-f255-4fca-b3c2-b743a9a9b14d admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.116424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795df3e-fb14-4dbc-b60a-9bde14d0e9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.117029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795df3e-fb14-4dbc-b60a-9bde14d0e9ee admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.117291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795df3e-fb14-4dbc-b60a-9bde14d0e9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.117565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795df3e-fb14-4dbc-b60a-9bde14d0e9ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.118269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795df3e-fb14-4dbc-b60a-9bde14d0e9ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:39.118561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795df3e-fb14-4dbc-b60a-9bde14d0e9ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.120845 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795df3e-fb14-4dbc-b60a-9bde14d0e9ee admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.135647 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 946/1873] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:39.135647 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795df3e-fb14-4dbc-b60a-9bde14d0e9ee admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.143811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795df3e-fb14-4dbc-b60a-9bde14d0e9ee admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.148476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c791a1-d2b7-497a-b327-2f8da91ee0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.149100 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c791a1-d2b7-497a-b327-2f8da91ee0b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.149374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c791a1-d2b7-497a-b327-2f8da91ee0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.149644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c791a1-d2b7-497a-b327-2f8da91ee0b3 admin admin] PATH_INFO: `/v3/projects/3f228bd771994ec6887d316f90ae91f6/users/09b9b5a046104932ae00e4be9cc9d69c/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.151346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 928/1874] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:39.152182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa381cc-74ff-4932-b4e1-1106d905c629 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.165170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa381cc-74ff-4932-b4e1-1106d905c629 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.165698 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c791a1-d2b7-497a-b327-2f8da91ee0b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f228bd771994ec6887d316f90ae91f6, user_id=09b9b5a046104932ae00e4be9cc9d69c, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.169300 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c791a1-d2b7-497a-b327-2f8da91ee0b3 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.170341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5079af-656d-45b6-97bc-41be9c0033b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.170933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5079af-656d-45b6-97bc-41be9c0033b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.171211 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5079af-656d-45b6-97bc-41be9c0033b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.171483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5079af-656d-45b6-97bc-41be9c0033b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.172108 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5079af-656d-45b6-97bc-41be9c0033b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.174104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5079af-656d-45b6-97bc-41be9c0033b7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.274971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 947/1875] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:39 2022] PUT /identity/v3/projects/3f228bd771994ec6887d316f90ae91f6/users/09b9b5a046104932ae00e4be9cc9d69c/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:39.276529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c791a1-d2b7-497a-b327-2f8da91ee0b3 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.315288 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 929/1876] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:39 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:39.316377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5079af-656d-45b6-97bc-41be9c0033b7 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.344703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c791a1-d2b7-497a-b327-2f8da91ee0b3 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.390662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5079af-656d-45b6-97bc-41be9c0033b7 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.396377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63faac4-b00e-4b2f-86e8-a356a58ca982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.396992 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63faac4-b00e-4b2f-86e8-a356a58ca982 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.398170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63faac4-b00e-4b2f-86e8-a356a58ca982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.398170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63faac4-b00e-4b2f-86e8-a356a58ca982 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.398917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63faac4-b00e-4b2f-86e8-a356a58ca982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.400512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63faac4-b00e-4b2f-86e8-a356a58ca982 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.410586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c29572-abfe-49ba-b3eb-9bc8f37a86b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.411369 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 930/1877] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:39.411996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63faac4-b00e-4b2f-86e8-a356a58ca982 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.413374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c29572-abfe-49ba-b3eb-9bc8f37a86b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.413694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c29572-abfe-49ba-b3eb-9bc8f37a86b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.414043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c29572-abfe-49ba-b3eb-9bc8f37a86b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.414689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c29572-abfe-49ba-b3eb-9bc8f37a86b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.418093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c29572-abfe-49ba-b3eb-9bc8f37a86b6 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.423633 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63faac4-b00e-4b2f-86e8-a356a58ca982 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.429316 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e723a0d-319a-44c1-a2e2-1100b43ff348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.429935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e723a0d-319a-44c1-a2e2-1100b43ff348 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.430220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e723a0d-319a-44c1-a2e2-1100b43ff348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.430525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e723a0d-319a-44c1-a2e2-1100b43ff348 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.431183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e723a0d-319a-44c1-a2e2-1100b43ff348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.433183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e723a0d-319a-44c1-a2e2-1100b43ff348 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.552996 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 948/1878] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:39 2022] POST /identity/v3/projects => generated 379 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:39.553802 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c29572-abfe-49ba-b3eb-9bc8f37a86b6 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.612688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 931/1879] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:39 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:39.612688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e723a0d-319a-44c1-a2e2-1100b43ff348 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.622577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e723a0d-319a-44c1-a2e2-1100b43ff348 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.627477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e613580-9a8a-4083-835d-5d214bd236bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.628087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e613580-9a8a-4083-835d-5d214bd236bd admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.628410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e613580-9a8a-4083-835d-5d214bd236bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.628688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e613580-9a8a-4083-835d-5d214bd236bd admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.628940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c29572-abfe-49ba-b3eb-9bc8f37a86b6 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.629320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e613580-9a8a-4083-835d-5d214bd236bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.631291 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e613580-9a8a-4083-835d-5d214bd236bd admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.633564 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de5413-3af1-4f25-bac4-ed1f2ce28ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.634331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de5413-3af1-4f25-bac4-ed1f2ce28ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.634668 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de5413-3af1-4f25-bac4-ed1f2ce28ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.634921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de5413-3af1-4f25-bac4-ed1f2ce28ddb admin admin] PATH_INFO: `/v3/projects/3f228bd771994ec6887d316f90ae91f6/users/09b9b5a046104932ae00e4be9cc9d69c/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.636884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de5413-3af1-4f25-bac4-ed1f2ce28ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f228bd771994ec6887d316f90ae91f6, user_id=09b9b5a046104932ae00e4be9cc9d69c, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.639547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de5413-3af1-4f25-bac4-ed1f2ce28ddb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.650188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 949/1880] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:39 2022] PUT /identity/v3/projects/3f228bd771994ec6887d316f90ae91f6/users/09b9b5a046104932ae00e4be9cc9d69c/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:39.651036 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de5413-3af1-4f25-bac4-ed1f2ce28ddb admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.714480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de5413-3af1-4f25-bac4-ed1f2ce28ddb admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.719194 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4005a05-1cbf-434e-8a29-28804372f2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.719795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4005a05-1cbf-434e-8a29-28804372f2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.720071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4005a05-1cbf-434e-8a29-28804372f2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.720369 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4005a05-1cbf-434e-8a29-28804372f2a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.721040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4005a05-1cbf-434e-8a29-28804372f2a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.722632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4005a05-1cbf-434e-8a29-28804372f2a3 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.726546 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 932/1881] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:39 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:39.727799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850542e5-e488-4ae8-b983-0e8b880b679f None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.729108 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850542e5-e488-4ae8-b983-0e8b880b679f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.729108 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850542e5-e488-4ae8-b983-0e8b880b679f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.744420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 950/1882] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:39.745463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4005a05-1cbf-434e-8a29-28804372f2a3 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.757529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4005a05-1cbf-434e-8a29-28804372f2a3 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.759003 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-850542e5-e488-4ae8-b983-0e8b880b679f None None] MFA Rules not processed for user `09b9b5a046104932ae00e4be9cc9d69c`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:39.760256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-850542e5-e488-4ae8-b983-0e8b880b679f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:39.762906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753c0cd-bafd-45ab-a984-0abf5a5aa7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.763505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753c0cd-bafd-45ab-a984-0abf5a5aa7f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.763780 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753c0cd-bafd-45ab-a984-0abf5a5aa7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.764053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753c0cd-bafd-45ab-a984-0abf5a5aa7f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.764713 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753c0cd-bafd-45ab-a984-0abf5a5aa7f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.766364 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753c0cd-bafd-45ab-a984-0abf5a5aa7f9 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.780550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 951/1883] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:39.780550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753c0cd-bafd-45ab-a984-0abf5a5aa7f9 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.790422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753c0cd-bafd-45ab-a984-0abf5a5aa7f9 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.797525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ac396-0547-491c-a67a-6a63f2b4e6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.798781 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ac396-0547-491c-a67a-6a63f2b4e6bd admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.799210 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ac396-0547-491c-a67a-6a63f2b4e6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.799359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ac396-0547-491c-a67a-6a63f2b4e6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.800456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ac396-0547-491c-a67a-6a63f2b4e6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.802718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ac396-0547-491c-a67a-6a63f2b4e6bd admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.814297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 952/1884] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:39.815164 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ac396-0547-491c-a67a-6a63f2b4e6bd admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.834258 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ac396-0547-491c-a67a-6a63f2b4e6bd admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.839598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539f5b4-8c69-4015-92e4-f8727c663ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.840303 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539f5b4-8c69-4015-92e4-f8727c663ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.840598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539f5b4-8c69-4015-92e4-f8727c663ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.840946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539f5b4-8c69-4015-92e4-f8727c663ab1 admin admin] PATH_INFO: `/v3/projects/b2a040ee218747f4a46046e948e9aa78/users/0f74943b5f264aa2924578ff53bf46ef/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.842378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 933/1885] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:39.843393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850542e5-e488-4ae8-b983-0e8b880b679f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.853760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850542e5-e488-4ae8-b983-0e8b880b679f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:39.858778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539f5b4-8c69-4015-92e4-f8727c663ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a040ee218747f4a46046e948e9aa78, user_id=0f74943b5f264aa2924578ff53bf46ef, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.861806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539f5b4-8c69-4015-92e4-f8727c663ab1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.865127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b4ec3-28b2-4427-b548-bbec543a4937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:39.866789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b4ec3-28b2-4427-b548-bbec543a4937 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:39.866943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b4ec3-28b2-4427-b548-bbec543a4937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:39.867272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b4ec3-28b2-4427-b548-bbec543a4937 admin admin] PATH_INFO: `/v3/projects/25f141f6504c4614ae7ea1e84c2cd187/users/e3a89ef321fc49e0ae114bdf8244925d/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:39.883608 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b4ec3-28b2-4427-b548-bbec543a4937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f141f6504c4614ae7ea1e84c2cd187, user_id=e3a89ef321fc49e0ae114bdf8244925d, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:39.886913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b4ec3-28b2-4427-b548-bbec543a4937 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:39.890974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:39.960699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 953/1886] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:39 2022] PUT /identity/v3/projects/b2a040ee218747f4a46046e948e9aa78/users/0f74943b5f264aa2924578ff53bf46ef/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:39.961531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539f5b4-8c69-4015-92e4-f8727c663ab1 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:39.963279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9539f5b4-8c69-4015-92e4-f8727c663ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:40.019248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 934/1887] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:39 2022] PUT /identity/v3/projects/25f141f6504c4614ae7ea1e84c2cd187/users/e3a89ef321fc49e0ae114bdf8244925d/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:40.019248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b4ec3-28b2-4427-b548-bbec543a4937 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:40.035289 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539f5b4-8c69-4015-92e4-f8727c663ab1 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:40.086450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b4ec3-28b2-4427-b548-bbec543a4937 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:40.087850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951785a1-f210-4a31-af11-04f060c47d53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:40.088497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951785a1-f210-4a31-af11-04f060c47d53 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:40.089316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951785a1-f210-4a31-af11-04f060c47d53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:40.089593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951785a1-f210-4a31-af11-04f060c47d53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:40.093668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11e910-0625-4005-aee1-d37efd28c5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:40.094875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11e910-0625-4005-aee1-d37efd28c5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:40.094875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11e910-0625-4005-aee1-d37efd28c5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:40.095050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11e910-0625-4005-aee1-d37efd28c5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:40.095617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11e910-0625-4005-aee1-d37efd28c5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:40.097196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11e910-0625-4005-aee1-d37efd28c5d9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:40.115153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 935/1888] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:40.115153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfd7aa-3e73-4a5c-8030-1ea2e648c1c4 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:40.115153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfd7aa-3e73-4a5c-8030-1ea2e648c1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:40.115153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfd7aa-3e73-4a5c-8030-1ea2e648c1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:40.146490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-38bfd7aa-3e73-4a5c-8030-1ea2e648c1c4 None None] MFA Rules not processed for user `e3a89ef321fc49e0ae114bdf8244925d`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:40.147752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-38bfd7aa-3e73-4a5c-8030-1ea2e648c1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:40.175683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951785a1-f210-4a31-af11-04f060c47d53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:40.175683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951785a1-f210-4a31-af11-04f060c47d53 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:40.189626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 954/1889] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:40.190492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951785a1-f210-4a31-af11-04f060c47d53 service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:40.203089 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951785a1-f210-4a31-af11-04f060c47d53 service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:40.208186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3f283-d0cd-4d32-a429-f342ec4e0193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:40.210029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3f283-d0cd-4d32-a429-f342ec4e0193 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:40.210316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3f283-d0cd-4d32-a429-f342ec4e0193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:40.210599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3f283-d0cd-4d32-a429-f342ec4e0193 admin admin] PATH_INFO: `/v3/projects/b2a040ee218747f4a46046e948e9aa78/users/0f74943b5f264aa2924578ff53bf46ef/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:40.212730 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3f283-d0cd-4d32-a429-f342ec4e0193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a040ee218747f4a46046e948e9aa78, user_id=0f74943b5f264aa2924578ff53bf46ef, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:40.216130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3f283-d0cd-4d32-a429-f342ec4e0193 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:40.251804 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 955/1890] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:40 2022] PUT /identity/v3/projects/b2a040ee218747f4a46046e948e9aa78/users/0f74943b5f264aa2924578ff53bf46ef/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:40.253072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b1f39-cd90-4e8c-a04e-43778154d290 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:40.253354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b1f39-cd90-4e8c-a04e-43778154d290 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:40.253626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b1f39-cd90-4e8c-a04e-43778154d290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:40.255449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 936/1891] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:40.259736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfd7aa-3e73-4a5c-8030-1ea2e648c1c4 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:40.288684 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfd7aa-3e73-4a5c-8030-1ea2e648c1c4 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:40.296866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-fc3b1f39-cd90-4e8c-a04e-43778154d290 None None] MFA Rules not processed for user `0f74943b5f264aa2924578ff53bf46ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:40.298335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-fc3b1f39-cd90-4e8c-a04e-43778154d290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:40.397035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd436b89-509a-418c-a7c7-93cf02c75d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:40.397660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd436b89-509a-418c-a7c7-93cf02c75d6e admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:40.397949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd436b89-509a-418c-a7c7-93cf02c75d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:40.398439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd436b89-509a-418c-a7c7-93cf02c75d6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:40.399976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd436b89-509a-418c-a7c7-93cf02c75d6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:40.403023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd436b89-509a-418c-a7c7-93cf02c75d6e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:40.438409 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 956/1892] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:40.473362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b1f39-cd90-4e8c-a04e-43778154d290 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:40.475547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-fc3b1f39-cd90-4e8c-a04e-43778154d290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:40.527671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 937/1893] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:40 2022] POST /identity/v3/projects => generated 363 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:27:40.533894 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd436b89-509a-418c-a7c7-93cf02c75d6e admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:40.561721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b1f39-cd90-4e8c-a04e-43778154d290 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:40.621668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd436b89-509a-418c-a7c7-93cf02c75d6e admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:40.628639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc1eb7-5f5a-4e10-ac62-5176577a984e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:40.635814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc1eb7-5f5a-4e10-ac62-5176577a984e admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:40.635814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc1eb7-5f5a-4e10-ac62-5176577a984e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:40.635814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc1eb7-5f5a-4e10-ac62-5176577a984e admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:40.635814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc1eb7-5f5a-4e10-ac62-5176577a984e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:40.637900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc1eb7-5f5a-4e10-ac62-5176577a984e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:40.641982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6afd5e-347d-402d-b544-c2fbd296fbb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:40.643161 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6afd5e-347d-402d-b544-c2fbd296fbb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:40.643628 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6afd5e-347d-402d-b544-c2fbd296fbb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:40.643960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6afd5e-347d-402d-b544-c2fbd296fbb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:40.718158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6afd5e-347d-402d-b544-c2fbd296fbb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:40.718944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6afd5e-347d-402d-b544-c2fbd296fbb8 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:40.729843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 957/1894] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:40.784968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 938/1895] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:40 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:27:40.789653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6afd5e-347d-402d-b544-c2fbd296fbb8 service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:40.806577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6afd5e-347d-402d-b544-c2fbd296fbb8 service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:40.820555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b16f8e-f652-47ac-b7b5-82d8de6a5f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:40.820555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b16f8e-f652-47ac-b7b5-82d8de6a5f77 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:40.820555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b16f8e-f652-47ac-b7b5-82d8de6a5f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:40.820555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b16f8e-f652-47ac-b7b5-82d8de6a5f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:40.820845 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b16f8e-f652-47ac-b7b5-82d8de6a5f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:40.822071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b16f8e-f652-47ac-b7b5-82d8de6a5f77 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:40.837745 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 958/1896] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:27:40.842477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc1eb7-5f5a-4e10-ac62-5176577a984e admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:40.859988 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc1eb7-5f5a-4e10-ac62-5176577a984e admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:40.869409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee0980-0e1c-4099-80c7-21b1a1ffbdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:40.871866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee0980-0e1c-4099-80c7-21b1a1ffbdab admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:40.871866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee0980-0e1c-4099-80c7-21b1a1ffbdab admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:40.871866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee0980-0e1c-4099-80c7-21b1a1ffbdab admin admin] PATH_INFO: `/v3/projects/0357f16c07694a8d984c6f592d04a61e/users/f3bdeae6ddc64eaf8751ed3b6e82a6b7/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:40.888007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee0980-0e1c-4099-80c7-21b1a1ffbdab admin admin] RBAC: Authorizing `identity:create_grant(project_id=0357f16c07694a8d984c6f592d04a61e, user_id=f3bdeae6ddc64eaf8751ed3b6e82a6b7, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:40.890834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee0980-0e1c-4099-80c7-21b1a1ffbdab admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:41.007515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b16f8e-f652-47ac-b7b5-82d8de6a5f77 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:41.018429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b16f8e-f652-47ac-b7b5-82d8de6a5f77 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:41.022258 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a7bff-33a6-4cc2-a6e1-e6e604e69e5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:41.022861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a7bff-33a6-4cc2-a6e1-e6e604e69e5e service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:41.023162 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a7bff-33a6-4cc2-a6e1-e6e604e69e5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:41.023435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a7bff-33a6-4cc2-a6e1-e6e604e69e5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:41.025310 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b89a7bff-33a6-4cc2-a6e1-e6e604e69e5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:41.048854 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 939/1897] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:40 2022] PUT /identity/v3/projects/0357f16c07694a8d984c6f592d04a61e/users/f3bdeae6ddc64eaf8751ed3b6e82a6b7/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:41.052658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee0980-0e1c-4099-80c7-21b1a1ffbdab admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:41.105914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a7bff-33a6-4cc2-a6e1-e6e604e69e5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:41.107408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a7bff-33a6-4cc2-a6e1-e6e604e69e5e service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:41.117811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 959/1898] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:27:41.125927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee0980-0e1c-4099-80c7-21b1a1ffbdab admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:41.131825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc8f6f-2b11-4930-9c21-3873c22fd7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:41.132548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc8f6f-2b11-4930-9c21-3873c22fd7e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:41.132826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc8f6f-2b11-4930-9c21-3873c22fd7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:41.133099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc8f6f-2b11-4930-9c21-3873c22fd7e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:41.133746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc8f6f-2b11-4930-9c21-3873c22fd7e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:41.135400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc8f6f-2b11-4930-9c21-3873c22fd7e0 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:41.147856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 940/1899] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:27:41.152122 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a7bff-33a6-4cc2-a6e1-e6e604e69e5e service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:41.167641 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a7bff-33a6-4cc2-a6e1-e6e604e69e5e service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:41.176360 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000b16a-e28b-49d2-ad81-a63996d82e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:41.177311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000b16a-e28b-49d2-ad81-a63996d82e8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:41.177655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000b16a-e28b-49d2-ad81-a63996d82e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:41.178001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000b16a-e28b-49d2-ad81-a63996d82e8a admin admin] PATH_INFO: `/v3/projects/0357f16c07694a8d984c6f592d04a61e/users/f3bdeae6ddc64eaf8751ed3b6e82a6b7/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:41.180215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a000b16a-e28b-49d2-ad81-a63996d82e8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0357f16c07694a8d984c6f592d04a61e, user_id=f3bdeae6ddc64eaf8751ed3b6e82a6b7, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:41.183082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a000b16a-e28b-49d2-ad81-a63996d82e8a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:41.194119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 960/1900] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:41 2022] PUT /identity/v3/projects/0357f16c07694a8d984c6f592d04a61e/users/f3bdeae6ddc64eaf8751ed3b6e82a6b7/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:41.213361 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943ef52-b011-42dd-b719-a613fa9037ab None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:41.214382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943ef52-b011-42dd-b719-a613fa9037ab None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:41.214826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943ef52-b011-42dd-b719-a613fa9037ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:41.257006 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7943ef52-b011-42dd-b719-a613fa9037ab None None] MFA Rules not processed for user `f3bdeae6ddc64eaf8751ed3b6e82a6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:41.258035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7943ef52-b011-42dd-b719-a613fa9037ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:41.269655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:41.357424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 941/1901] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:41.722238 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000b16a-e28b-49d2-ad81-a63996d82e8a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:41.778626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000b16a-e28b-49d2-ad81-a63996d82e8a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:41.782467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4005a05-c0dd-4761-bcc9-18f850a83c6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:41.783172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4005a05-c0dd-4761-bcc9-18f850a83c6b service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:41.783451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4005a05-c0dd-4761-bcc9-18f850a83c6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:41.783720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4005a05-c0dd-4761-bcc9-18f850a83c6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:41.793784 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4005a05-c0dd-4761-bcc9-18f850a83c6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:41.795219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4005a05-c0dd-4761-bcc9-18f850a83c6b service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:41.806087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 961/1902] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:42.360687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943ef52-b011-42dd-b719-a613fa9037ab None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:42.373015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943ef52-b011-42dd-b719-a613fa9037ab None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:42.376679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40fa14-a904-4c31-a9c2-5581cbc81ab0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:42.377232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40fa14-a904-4c31-a9c2-5581cbc81ab0 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:42.377513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40fa14-a904-4c31-a9c2-5581cbc81ab0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:42.377788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40fa14-a904-4c31-a9c2-5581cbc81ab0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:42.446445 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40fa14-a904-4c31-a9c2-5581cbc81ab0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:42.449163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40fa14-a904-4c31-a9c2-5581cbc81ab0 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:42.462001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 942/1903] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:43.070413 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:43.083772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4005a05-c0dd-4761-bcc9-18f850a83c6b service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:43.092229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4005a05-c0dd-4761-bcc9-18f850a83c6b service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:43.095801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812a898-e66e-4479-ab2b-17c0f6282d88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:43.096436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812a898-e66e-4479-ab2b-17c0f6282d88 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:43.097356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812a898-e66e-4479-ab2b-17c0f6282d88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:43.097639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812a898-e66e-4479-ab2b-17c0f6282d88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:43.099373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7812a898-e66e-4479-ab2b-17c0f6282d88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:43.160865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812a898-e66e-4479-ab2b-17c0f6282d88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:43.162378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812a898-e66e-4479-ab2b-17c0f6282d88 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:43.172336 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 962/1904] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:43.564063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40fa14-a904-4c31-a9c2-5581cbc81ab0 service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:43.565711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-bd40fa14-a904-4c31-a9c2-5581cbc81ab0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:43.585971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fe040-2d50-47c3-ae6e-5eecbddb9177 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:43.586248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fe040-2d50-47c3-ae6e-5eecbddb9177 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:43.586737 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fe040-2d50-47c3-ae6e-5eecbddb9177 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:43.588719 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 963/1905] 10.222.0.85 () {54 vars in 921 bytes} [Mon Aug 29 20:27:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:43.596120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed4e49-8f4f-449c-9c23-fa64599ff89e None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:43.596120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed4e49-8f4f-449c-9c23-fa64599ff89e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:43.596422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed4e49-8f4f-449c-9c23-fa64599ff89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:43.636801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-12ed4e49-8f4f-449c-9c23-fa64599ff89e None None] MFA Rules not processed for user `805df3047fee45c7a702d72a5c082eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:43.638268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-12ed4e49-8f4f-449c-9c23-fa64599ff89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:43.656792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40fa14-a904-4c31-a9c2-5581cbc81ab0 service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:43.662549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:43.663419 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:43.663765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:43.664485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] PATH_INFO: `/v3/users/bf0ed986e41b4f60ae9234c545c24880` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:43.665417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf0ed986e41b4f60ae9234c545c24880)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:43.668309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:43.734061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 964/1906] 10.222.0.85 () {58 vars in 1018 bytes} [Mon Aug 29 20:27:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:43.759446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed4e49-8f4f-449c-9c23-fa64599ff89e None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:43.761240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-12ed4e49-8f4f-449c-9c23-fa64599ff89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:43.775366 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed4e49-8f4f-449c-9c23-fa64599ff89e None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:43.782396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b3a52-8e74-4737-814b-a819e41386f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:43.783022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b3a52-8e74-4737-814b-a819e41386f1 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:43.783655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b3a52-8e74-4737-814b-a819e41386f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:43.783655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b3a52-8e74-4737-814b-a819e41386f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:43.793899 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b3a52-8e74-4737-814b-a819e41386f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:43.795775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b3a52-8e74-4737-814b-a819e41386f1 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:43.807142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 965/1907] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:44.000213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf0ed986e41b4f60ae9234c545c24880'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:44.000864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf0ed986e41b4f60ae9234c545c24880'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:44.013656 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf0ed986e41b4f60ae9234c545c24880'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:44.229251 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf0ed986e41b4f60ae9234c545c24880'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:44.229905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf0ed986e41b4f60ae9234c545c24880'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:44.253583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 943/1908] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:43 2022] DELETE /identity/v3/users/bf0ed986e41b4f60ae9234c545c24880 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:44.255281 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6649b-f32b-407d-8e5b-4ac7465b7dcc None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:44.255764 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6649b-f32b-407d-8e5b-4ac7465b7dcc None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:44.256023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6649b-f32b-407d-8e5b-4ac7465b7dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:44.281354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:44.288698 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-50d6649b-f32b-407d-8e5b-4ac7465b7dcc None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:44.291378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-50d6649b-f32b-407d-8e5b-4ac7465b7dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:44.400138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 966/1909] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:44.408809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:44.410512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:44.498004 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f25847-1718-454f-89fc-65215b889ea7 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:44.501970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42bed4-88af-4752-9d45-130dec097bcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:44.502591 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42bed4-88af-4752-9d45-130dec097bcc service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:44.502950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42bed4-88af-4752-9d45-130dec097bcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:44.503220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42bed4-88af-4752-9d45-130dec097bcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:44.520412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42bed4-88af-4752-9d45-130dec097bcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:44.521995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42bed4-88af-4752-9d45-130dec097bcc service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:44.534996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 944/1910] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:45.193913 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6649b-f32b-407d-8e5b-4ac7465b7dcc None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:45.196427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-50d6649b-f32b-407d-8e5b-4ac7465b7dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:45.219895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6649b-f32b-407d-8e5b-4ac7465b7dcc None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:45.234187 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47336174-d58a-443b-877e-e11930393009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:45.234187 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47336174-d58a-443b-877e-e11930393009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:45.234187 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47336174-d58a-443b-877e-e11930393009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:45.234418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47336174-d58a-443b-877e-e11930393009 admin admin] PATH_INFO: `/v3/projects/45feb051b10140e396339f89ffa2cb36` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:45.235505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47336174-d58a-443b-877e-e11930393009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45feb051b10140e396339f89ffa2cb36)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:45.238088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47336174-d58a-443b-877e-e11930393009 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:45.388617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-47336174-d58a-443b-877e-e11930393009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45feb051b10140e396339f89ffa2cb36', 'request_id': 'req-47336174-d58a-443b-877e-e11930393009'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:45.397088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-47336174-d58a-443b-877e-e11930393009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45feb051b10140e396339f89ffa2cb36', 'request_id': 'req-47336174-d58a-443b-877e-e11930393009'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:45.402989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-47336174-d58a-443b-877e-e11930393009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45feb051b10140e396339f89ffa2cb36', 'request_id': 'req-47336174-d58a-443b-877e-e11930393009'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:45.534881 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-47336174-d58a-443b-877e-e11930393009 admin admin] The token cache is being invalidate because project 45feb051b10140e396339f89ffa2cb36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:45.534881 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-47336174-d58a-443b-877e-e11930393009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:45.534881 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-47336174-d58a-443b-877e-e11930393009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:45.539000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 967/1911] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:45 2022] DELETE /identity/v3/projects/45feb051b10140e396339f89ffa2cb36 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:45.542973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42bed4-88af-4752-9d45-130dec097bcc service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:45.555778 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42bed4-88af-4752-9d45-130dec097bcc service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:45.641315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c3ff2-1d91-4c26-802b-c37419496fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:45.641315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c3ff2-1d91-4c26-802b-c37419496fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:45.641315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c3ff2-1d91-4c26-802b-c37419496fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:45.641315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c3ff2-1d91-4c26-802b-c37419496fb8 admin admin] PATH_INFO: `/v3/users/7d253c5f6e8f4af980ba547a445e7028` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:45.642231 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c3ff2-1d91-4c26-802b-c37419496fb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d253c5f6e8f4af980ba547a445e7028)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:45.642231 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c3ff2-1d91-4c26-802b-c37419496fb8 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:45.837394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-596c3ff2-1d91-4c26-802b-c37419496fb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d253c5f6e8f4af980ba547a445e7028'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:45.838039 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-596c3ff2-1d91-4c26-802b-c37419496fb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d253c5f6e8f4af980ba547a445e7028'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:45.845582 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-596c3ff2-1d91-4c26-802b-c37419496fb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d253c5f6e8f4af980ba547a445e7028'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:45.851726 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:46.012510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-596c3ff2-1d91-4c26-802b-c37419496fb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d253c5f6e8f4af980ba547a445e7028'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:46.013490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-596c3ff2-1d91-4c26-802b-c37419496fb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d253c5f6e8f4af980ba547a445e7028'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:46.042510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 945/1912] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:45 2022] DELETE /identity/v3/users/7d253c5f6e8f4af980ba547a445e7028 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:46.717102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47336174-d58a-443b-877e-e11930393009 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:46.718843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-47336174-d58a-443b-877e-e11930393009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:46.731746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47336174-d58a-443b-877e-e11930393009 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:46.816275 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:46.817615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:46.818158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:46.819361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038 admin admin] PATH_INFO: `/v3/projects/cfd66a74ff6b4958a9e9d5751c6743d4` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:46.820488 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfd66a74ff6b4958a9e9d5751c6743d4)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:46.822625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:47.002134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfd66a74ff6b4958a9e9d5751c6743d4', 'request_id': 'req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:47.008962 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfd66a74ff6b4958a9e9d5751c6743d4', 'request_id': 'req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:47.015329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfd66a74ff6b4958a9e9d5751c6743d4', 'request_id': 'req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:47.200955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038 admin admin] The token cache is being invalidate because project cfd66a74ff6b4958a9e9d5751c6743d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:47.201777 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:47.201909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-189e8799-1b5e-4ed5-a3ea-d17d9e38b038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:47.203767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 968/1913] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:46 2022] DELETE /identity/v3/projects/cfd66a74ff6b4958a9e9d5751c6743d4 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:47.208710 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90dbde-0991-49f2-b9a1-abfaa3e60d2b None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:47.209207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90dbde-0991-49f2-b9a1-abfaa3e60d2b None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:47.209284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90dbde-0991-49f2-b9a1-abfaa3e60d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:47.229466 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-4b90dbde-0991-49f2-b9a1-abfaa3e60d2b None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:47.230698 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4b90dbde-0991-49f2-b9a1-abfaa3e60d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:47.270288 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:47.307698 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 946/1914] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:47.315814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc7918-23d6-45f0-b097-f454704b8908 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:47.316109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc7918-23d6-45f0-b097-f454704b8908 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:47.316446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc7918-23d6-45f0-b097-f454704b8908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:47.337177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-06bc7918-23d6-45f0-b097-f454704b8908 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:47.338527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-06bc7918-23d6-45f0-b097-f454704b8908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:47.347124 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 969/1915] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:47.351316 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90dbde-0991-49f2-b9a1-abfaa3e60d2b None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:47.365937 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90dbde-0991-49f2-b9a1-abfaa3e60d2b None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:47.373850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32186e4d-9614-4c5a-8c88-6497b2d9855d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:47.374470 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32186e4d-9614-4c5a-8c88-6497b2d9855d admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:47.374753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32186e4d-9614-4c5a-8c88-6497b2d9855d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:47.375030 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32186e4d-9614-4c5a-8c88-6497b2d9855d admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:47.375687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32186e4d-9614-4c5a-8c88-6497b2d9855d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:47.375989 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32186e4d-9614-4c5a-8c88-6497b2d9855d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:47.377617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32186e4d-9614-4c5a-8c88-6497b2d9855d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:47.392675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 947/1916] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:47.396454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc7918-23d6-45f0-b097-f454704b8908 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:47.406237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc7918-23d6-45f0-b097-f454704b8908 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:47.411625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73973e3-1a2a-4469-bf71-ed11c7f2c13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:47.412269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73973e3-1a2a-4469-bf71-ed11c7f2c13a admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:47.413134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73973e3-1a2a-4469-bf71-ed11c7f2c13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:47.413134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73973e3-1a2a-4469-bf71-ed11c7f2c13a admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:47.413445 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73973e3-1a2a-4469-bf71-ed11c7f2c13a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:47.415570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73973e3-1a2a-4469-bf71-ed11c7f2c13a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:47.523055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 970/1917] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:47 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:47.526901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32186e4d-9614-4c5a-8c88-6497b2d9855d admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:47.549134 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32186e4d-9614-4c5a-8c88-6497b2d9855d admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:47.646989 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd13b34-8d7f-4a99-9040-a9f3e232789b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:47.647526 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd13b34-8d7f-4a99-9040-a9f3e232789b admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:47.647826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd13b34-8d7f-4a99-9040-a9f3e232789b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:47.648072 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd13b34-8d7f-4a99-9040-a9f3e232789b admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:47.648748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd13b34-8d7f-4a99-9040-a9f3e232789b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:47.650823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd13b34-8d7f-4a99-9040-a9f3e232789b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:47.792691 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 948/1918] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:47 2022] POST /identity/v3/users => generated 294 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:27:47.795106 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73973e3-1a2a-4469-bf71-ed11c7f2c13a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:47.806389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73973e3-1a2a-4469-bf71-ed11c7f2c13a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:47.811261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492a9f5-cc22-4fa0-b018-758e1613616d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:47.811951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492a9f5-cc22-4fa0-b018-758e1613616d admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:47.812331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492a9f5-cc22-4fa0-b018-758e1613616d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:47.812616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492a9f5-cc22-4fa0-b018-758e1613616d admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:47.813339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492a9f5-cc22-4fa0-b018-758e1613616d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:47.814952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492a9f5-cc22-4fa0-b018-758e1613616d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:47.826688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 971/1919] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:47.826826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd13b34-8d7f-4a99-9040-a9f3e232789b admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:47.828718 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-dbd13b34-8d7f-4a99-9040-a9f3e232789b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:47.830343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492a9f5-cc22-4fa0-b018-758e1613616d admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:47.842498 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd13b34-8d7f-4a99-9040-a9f3e232789b admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:47.842736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492a9f5-cc22-4fa0-b018-758e1613616d admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:47.847332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b4ec9-13bb-4d17-8d6c-a8fde358fe47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:47.848028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766aa84-faf2-4445-94b2-d31de3895995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:47.848362 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b4ec9-13bb-4d17-8d6c-a8fde358fe47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:47.855098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766aa84-faf2-4445-94b2-d31de3895995 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:47.855230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b4ec9-13bb-4d17-8d6c-a8fde358fe47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:47.855323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766aa84-faf2-4445-94b2-d31de3895995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:47.855408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b4ec9-13bb-4d17-8d6c-a8fde358fe47 admin admin] PATH_INFO: `/v3/users/bcd23774168849c4b4f1e70b42cfe6f8` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:47.855492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766aa84-faf2-4445-94b2-d31de3895995 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:47.855492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766aa84-faf2-4445-94b2-d31de3895995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:47.855662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b4ec9-13bb-4d17-8d6c-a8fde358fe47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd23774168849c4b4f1e70b42cfe6f8)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:47.855745 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766aa84-faf2-4445-94b2-d31de3895995 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:47.855828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b4ec9-13bb-4d17-8d6c-a8fde358fe47 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:47.863957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 972/1920] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:47.868267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766aa84-faf2-4445-94b2-d31de3895995 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:47.882357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766aa84-faf2-4445-94b2-d31de3895995 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:47.890323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313b8dd-f7ea-4767-87c5-18e8bc0ddb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:47.891235 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313b8dd-f7ea-4767-87c5-18e8bc0ddb13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:47.891684 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313b8dd-f7ea-4767-87c5-18e8bc0ddb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:47.892398 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313b8dd-f7ea-4767-87c5-18e8bc0ddb13 admin admin] PATH_INFO: `/v3/projects/419f9789406045e7ab353e50cdc2fce7/users/2a2e74ddb58b4f8690ea1e562bc34389/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:47.922184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313b8dd-f7ea-4767-87c5-18e8bc0ddb13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=419f9789406045e7ab353e50cdc2fce7, user_id=2a2e74ddb58b4f8690ea1e562bc34389, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:47.927127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313b8dd-f7ea-4767-87c5-18e8bc0ddb13 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:48.034975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 973/1921] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:47 2022] PUT /identity/v3/projects/419f9789406045e7ab353e50cdc2fce7/users/2a2e74ddb58b4f8690ea1e562bc34389/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:48.042031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823093d-4c82-4320-96a0-5093f2a8d5dc None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:48.042517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823093d-4c82-4320-96a0-5093f2a8d5dc None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:48.042985 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823093d-4c82-4320-96a0-5093f2a8d5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:48.087991 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e823093d-4c82-4320-96a0-5093f2a8d5dc None None] MFA Rules not processed for user `2a2e74ddb58b4f8690ea1e562bc34389`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:48.090150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e823093d-4c82-4320-96a0-5093f2a8d5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:48.093216 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e07b4ec9-13bb-4d17-8d6c-a8fde358fe47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd23774168849c4b4f1e70b42cfe6f8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:48.093879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e07b4ec9-13bb-4d17-8d6c-a8fde358fe47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd23774168849c4b4f1e70b42cfe6f8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:48.102266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e07b4ec9-13bb-4d17-8d6c-a8fde358fe47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd23774168849c4b4f1e70b42cfe6f8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:48.165851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 974/1922] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:48.170267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823093d-4c82-4320-96a0-5093f2a8d5dc None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:48.194967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e07b4ec9-13bb-4d17-8d6c-a8fde358fe47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd23774168849c4b4f1e70b42cfe6f8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:48.195430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e07b4ec9-13bb-4d17-8d6c-a8fde358fe47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd23774168849c4b4f1e70b42cfe6f8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:48.218475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 949/1923] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:47 2022] DELETE /identity/v3/users/bcd23774168849c4b4f1e70b42cfe6f8 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:48.223745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ad1b4-70c8-4349-bc27-bb240c4e9b77 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:48.223981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ad1b4-70c8-4349-bc27-bb240c4e9b77 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:48.224465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ad1b4-70c8-4349-bc27-bb240c4e9b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:48.241570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823093d-4c82-4320-96a0-5093f2a8d5dc None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:48.244383 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f12ad1b4-70c8-4349-bc27-bb240c4e9b77 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:48.245641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f12ad1b4-70c8-4349-bc27-bb240c4e9b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:48.316619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 950/1924] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:48.328859 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ad1b4-70c8-4349-bc27-bb240c4e9b77 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:48.330746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f12ad1b4-70c8-4349-bc27-bb240c4e9b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:48.352065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aa275-e506-46e8-8204-0fdd3e13824c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:48.353745 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa275-e506-46e8-8204-0fdd3e13824c admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:48.354030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa275-e506-46e8-8204-0fdd3e13824c admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:48.354377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa275-e506-46e8-8204-0fdd3e13824c admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:48.358974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816aa275-e506-46e8-8204-0fdd3e13824c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:48.361431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816aa275-e506-46e8-8204-0fdd3e13824c admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:48.405190 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ad1b4-70c8-4349-bc27-bb240c4e9b77 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:48.409057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4cd41-f409-467f-9aaa-cc0927c59b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:48.409691 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4cd41-f409-467f-9aaa-cc0927c59b27 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:48.409967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4cd41-f409-467f-9aaa-cc0927c59b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:48.410241 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4cd41-f409-467f-9aaa-cc0927c59b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:48.420237 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4cd41-f409-467f-9aaa-cc0927c59b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:48.421684 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4cd41-f409-467f-9aaa-cc0927c59b27 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:48.433400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 951/1925] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:48.459442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 975/1926] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:48 2022] POST /identity/v3/projects => generated 387 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:27:48.462089 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4cd41-f409-467f-9aaa-cc0927c59b27 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:48.464201 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-42a4cd41-f409-467f-9aaa-cc0927c59b27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:48.477900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4cd41-f409-467f-9aaa-cc0927c59b27 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:48.546224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:48.548328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00b7ff-c2b2-4e94-ac75-902179f79760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:48.548950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00b7ff-c2b2-4e94-ac75-902179f79760 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:48.549331 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00b7ff-c2b2-4e94-ac75-902179f79760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:48.549879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00b7ff-c2b2-4e94-ac75-902179f79760 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:48.551079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00b7ff-c2b2-4e94-ac75-902179f79760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:48.553304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00b7ff-c2b2-4e94-ac75-902179f79760 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:48.637594 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 952/1927] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:48 2022] POST /identity/v3/users => generated 300 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:48.641482 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aa275-e506-46e8-8204-0fdd3e13824c admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:48.656991 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aa275-e506-46e8-8204-0fdd3e13824c admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:48.675624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce1a48-7471-48ac-80ea-acb6792545b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:48.676737 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce1a48-7471-48ac-80ea-acb6792545b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:48.677021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce1a48-7471-48ac-80ea-acb6792545b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:48.677298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce1a48-7471-48ac-80ea-acb6792545b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:48.678075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce1a48-7471-48ac-80ea-acb6792545b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:48.679587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce1a48-7471-48ac-80ea-acb6792545b1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:48.692800 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 976/1928] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:48.697641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00b7ff-c2b2-4e94-ac75-902179f79760 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:48.717629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00b7ff-c2b2-4e94-ac75-902179f79760 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:48.723559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb5a11-9cdc-4843-b815-5ae366378ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:48.724309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb5a11-9cdc-4843-b815-5ae366378ae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:48.724592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb5a11-9cdc-4843-b815-5ae366378ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:48.724869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb5a11-9cdc-4843-b815-5ae366378ae8 admin admin] PATH_INFO: `/v3/projects/6d60ed8b04ca46cc8b67d4c55af57b6d/users/019ad73a11464e9cb7a8c38b6d2fcfbe/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:48.744330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb5a11-9cdc-4843-b815-5ae366378ae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d60ed8b04ca46cc8b67d4c55af57b6d, user_id=019ad73a11464e9cb7a8c38b6d2fcfbe, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:48.747194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb5a11-9cdc-4843-b815-5ae366378ae8 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:48.852796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 953/1929] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:48 2022] PUT /identity/v3/projects/6d60ed8b04ca46cc8b67d4c55af57b6d/users/019ad73a11464e9cb7a8c38b6d2fcfbe/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:48.857229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce1a48-7471-48ac-80ea-acb6792545b1 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:48.868848 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce1a48-7471-48ac-80ea-acb6792545b1 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:48.933528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee9e9a-880f-47d5-9b3b-639c2aaa401a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:48.934237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee9e9a-880f-47d5-9b3b-639c2aaa401a admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:48.934758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee9e9a-880f-47d5-9b3b-639c2aaa401a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:48.935107 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee9e9a-880f-47d5-9b3b-639c2aaa401a admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:48.935837 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee9e9a-880f-47d5-9b3b-639c2aaa401a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:48.937567 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee9e9a-880f-47d5-9b3b-639c2aaa401a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:48.952912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 977/1930] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:48.954333 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb5a11-9cdc-4843-b815-5ae366378ae8 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:48.973235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb5a11-9cdc-4843-b815-5ae366378ae8 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:48.974530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee9e9a-880f-47d5-9b3b-639c2aaa401a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:48.978289 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-18ee9e9a-880f-47d5-9b3b-639c2aaa401a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:48.979900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924df2b-6851-4b95-8e75-c7933ecd853f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:48.979900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924df2b-6851-4b95-8e75-c7933ecd853f admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:48.980510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924df2b-6851-4b95-8e75-c7933ecd853f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:48.981163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924df2b-6851-4b95-8e75-c7933ecd853f admin admin] PATH_INFO: `/v3/projects/6d60ed8b04ca46cc8b67d4c55af57b6d/users/019ad73a11464e9cb7a8c38b6d2fcfbe/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:48.983525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924df2b-6851-4b95-8e75-c7933ecd853f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d60ed8b04ca46cc8b67d4c55af57b6d, user_id=019ad73a11464e9cb7a8c38b6d2fcfbe, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:48.986163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924df2b-6851-4b95-8e75-c7933ecd853f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:48.993578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee9e9a-880f-47d5-9b3b-639c2aaa401a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:48.998218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 954/1931] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:48 2022] PUT /identity/v3/projects/6d60ed8b04ca46cc8b67d4c55af57b6d/users/019ad73a11464e9cb7a8c38b6d2fcfbe/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:49.011482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ae427-2118-4e66-bd3d-e9d4ee6881a5 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:49.011985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ae427-2118-4e66-bd3d-e9d4ee6881a5 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:49.012657 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ae427-2118-4e66-bd3d-e9d4ee6881a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:49.043871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-2f4ae427-2118-4e66-bd3d-e9d4ee6881a5 None None] MFA Rules not processed for user `019ad73a11464e9cb7a8c38b6d2fcfbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:49.045287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2f4ae427-2118-4e66-bd3d-e9d4ee6881a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:49.086823 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:49.087457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:49.087736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:49.088007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] PATH_INFO: `/v3/projects/43c3b64ca1bc4f219042011d1c8a5b52` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:49.089230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c3b64ca1bc4f219042011d1c8a5b52)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:49.091236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:49.116772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 955/1932] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:49.117626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ae427-2118-4e66-bd3d-e9d4ee6881a5 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:49.118874 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2f4ae427-2118-4e66-bd3d-e9d4ee6881a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:49.175314 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ae427-2118-4e66-bd3d-e9d4ee6881a5 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:49.178972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe04a10-6c2e-4bfd-bbe2-a3bf4d0f1bc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:49.179580 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe04a10-6c2e-4bfd-bbe2-a3bf4d0f1bc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:49.179864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe04a10-6c2e-4bfd-bbe2-a3bf4d0f1bc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:49.180141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe04a10-6c2e-4bfd-bbe2-a3bf4d0f1bc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:49.181970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-cbe04a10-6c2e-4bfd-bbe2-a3bf4d0f1bc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:49.216627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c3b64ca1bc4f219042011d1c8a5b52', 'request_id': 'req-ea826f2d-268d-46ea-889f-29561950acf2'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:49.223215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c3b64ca1bc4f219042011d1c8a5b52', 'request_id': 'req-ea826f2d-268d-46ea-889f-29561950acf2'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:49.229212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c3b64ca1bc4f219042011d1c8a5b52', 'request_id': 'req-ea826f2d-268d-46ea-889f-29561950acf2'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:49.241343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe04a10-6c2e-4bfd-bbe2-a3bf4d0f1bc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:49.242791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe04a10-6c2e-4bfd-bbe2-a3bf4d0f1bc6 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:49.252035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 956/1933] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:49.253472 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe04a10-6c2e-4bfd-bbe2-a3bf4d0f1bc6 service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:49.254842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-cbe04a10-6c2e-4bfd-bbe2-a3bf4d0f1bc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:49.265977 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe04a10-6c2e-4bfd-bbe2-a3bf4d0f1bc6 service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:49.318372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ac3c1-1505-48e7-91dc-722c04956e59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:49.319309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ac3c1-1505-48e7-91dc-722c04956e59 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:49.319595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ac3c1-1505-48e7-91dc-722c04956e59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:49.319872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ac3c1-1505-48e7-91dc-722c04956e59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:49.328903 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] The token cache is being invalidate because project 43c3b64ca1bc4f219042011d1c8a5b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:49.329457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:49.330253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:49.333060 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 978/1934] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:48 2022] DELETE /identity/v3/projects/43c3b64ca1bc4f219042011d1c8a5b52 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:49.337343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:49.339456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:49.399759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ac3c1-1505-48e7-91dc-722c04956e59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:49.401906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ac3c1-1505-48e7-91dc-722c04956e59 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:49.412905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 957/1935] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:49.420930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea826f2d-268d-46ea-889f-29561950acf2 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:49.425885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cebe6-add5-4b38-bf11-1739f0335b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:49.428506 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cebe6-add5-4b38-bf11-1739f0335b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:49.429013 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cebe6-add5-4b38-bf11-1739f0335b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:49.429466 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cebe6-add5-4b38-bf11-1739f0335b7b admin admin] PATH_INFO: `/v3/users/b77e2f374d3c4e73897ff358681bdb74` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:49.430943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cebe6-add5-4b38-bf11-1739f0335b7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b77e2f374d3c4e73897ff358681bdb74)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:49.433443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cebe6-add5-4b38-bf11-1739f0335b7b admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:49.549354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:49.611649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ac3c1-1505-48e7-91dc-722c04956e59 service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:49.629550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ac3c1-1505-48e7-91dc-722c04956e59 service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:49.636107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:49.636780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:49.637063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:49.637357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] PATH_INFO: `/v3/users/2a2e74ddb58b4f8690ea1e562bc34389` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:49.638509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a2e74ddb58b4f8690ea1e562bc34389)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:49.640652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:49.728001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d12cebe6-add5-4b38-bf11-1739f0335b7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b77e2f374d3c4e73897ff358681bdb74'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:49.734687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d12cebe6-add5-4b38-bf11-1739f0335b7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b77e2f374d3c4e73897ff358681bdb74'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:49.863590 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d12cebe6-add5-4b38-bf11-1739f0335b7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b77e2f374d3c4e73897ff358681bdb74'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:49.864370 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d12cebe6-add5-4b38-bf11-1739f0335b7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b77e2f374d3c4e73897ff358681bdb74'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:49.879092 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d12cebe6-add5-4b38-bf11-1739f0335b7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b77e2f374d3c4e73897ff358681bdb74'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:49.880033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a2e74ddb58b4f8690ea1e562bc34389'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:49.880784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a2e74ddb58b4f8690ea1e562bc34389'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:49.886141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 979/1936] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:49 2022] DELETE /identity/v3/users/b77e2f374d3c4e73897ff358681bdb74 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:49.889167 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a2e74ddb58b4f8690ea1e562bc34389'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:50.012804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a2e74ddb58b4f8690ea1e562bc34389'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:50.013665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a2e74ddb58b4f8690ea1e562bc34389'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:50.033021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 958/1937] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:49 2022] DELETE /identity/v3/users/2a2e74ddb58b4f8690ea1e562bc34389 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:50.037652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064e0f6-3048-43d8-91d8-3701b1761a1c None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:50.038018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064e0f6-3048-43d8-91d8-3701b1761a1c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:50.038388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064e0f6-3048-43d8-91d8-3701b1761a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:50.058296 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d064e0f6-3048-43d8-91d8-3701b1761a1c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:50.059507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d064e0f6-3048-43d8-91d8-3701b1761a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:50.127032 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 980/1938] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:50.137474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:50.138977 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:50.197364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af196ffd-47e8-4c30-87a1-973fbf03a107 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:50.202700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a2600-2d18-43a6-a15d-181a110caa69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:50.202939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a2600-2d18-43a6-a15d-181a110caa69 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:50.203318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a2600-2d18-43a6-a15d-181a110caa69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:50.203663 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a2600-2d18-43a6-a15d-181a110caa69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:50.212814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a2600-2d18-43a6-a15d-181a110caa69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:50.214305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a2600-2d18-43a6-a15d-181a110caa69 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:50.223615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 959/1939] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:50.305539 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064e0f6-3048-43d8-91d8-3701b1761a1c None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:50.307206 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d064e0f6-3048-43d8-91d8-3701b1761a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:50.322622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064e0f6-3048-43d8-91d8-3701b1761a1c None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:50.327873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:50.330368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:50.330368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:50.330368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] PATH_INFO: `/v3/projects/3093f53915e34cb4a96ab1acc3fdba2b` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:50.331128 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3093f53915e34cb4a96ab1acc3fdba2b)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:50.333498 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:50.476283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3093f53915e34cb4a96ab1acc3fdba2b', 'request_id': 'req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:50.482658 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3093f53915e34cb4a96ab1acc3fdba2b', 'request_id': 'req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:50.488122 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3093f53915e34cb4a96ab1acc3fdba2b', 'request_id': 'req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:50.565945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] The token cache is being invalidate because project 3093f53915e34cb4a96ab1acc3fdba2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:50.566241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:50.566924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:50.571592 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 981/1940] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:50 2022] DELETE /identity/v3/projects/3093f53915e34cb4a96ab1acc3fdba2b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:50.573701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a2600-2d18-43a6-a15d-181a110caa69 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:50.584607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a2600-2d18-43a6-a15d-181a110caa69 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:50.596440 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:50.676881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:50.677708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:50.678097 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:50.678594 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] PATH_INFO: `/v3/users/0f74943b5f264aa2924578ff53bf46ef` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:50.688604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f74943b5f264aa2924578ff53bf46ef)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:50.691230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:50.831336 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:50.832916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:50.846353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff251ee-f09c-46ac-a4c1-f99dcc29b2d0 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:50.851659 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:50.857306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:50.857306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:50.857306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] PATH_INFO: `/v3/projects/419f9789406045e7ab353e50cdc2fce7` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:50.857306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=419f9789406045e7ab353e50cdc2fce7)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:50.857306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:50.998111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f74943b5f264aa2924578ff53bf46ef'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:50.998794 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f74943b5f264aa2924578ff53bf46ef'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.005912 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f74943b5f264aa2924578ff53bf46ef'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.069627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '419f9789406045e7ab353e50cdc2fce7', 'request_id': 'req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.076222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '419f9789406045e7ab353e50cdc2fce7', 'request_id': 'req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.081761 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '419f9789406045e7ab353e50cdc2fce7', 'request_id': 'req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.125923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f74943b5f264aa2924578ff53bf46ef'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.126540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f74943b5f264aa2924578ff53bf46ef'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.144764 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 960/1941] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:50 2022] DELETE /identity/v3/users/0f74943b5f264aa2924578ff53bf46ef => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:51.199704 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] The token cache is being invalidate because project 419f9789406045e7ab353e50cdc2fce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:51.200268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.200911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.202424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 982/1942] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:50 2022] DELETE /identity/v3/projects/419f9789406045e7ab353e50cdc2fce7 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:51.206846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:51.208259 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:51.275408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2c250-d7de-4241-980a-d2305ce414c3 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:51.281076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8e9bb-b4c6-43db-8c22-7778c47a5672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:51.281737 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8e9bb-b4c6-43db-8c22-7778c47a5672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:51.282016 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8e9bb-b4c6-43db-8c22-7778c47a5672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:51.282292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8e9bb-b4c6-43db-8c22-7778c47a5672 admin admin] PATH_INFO: `/v3/users/019ad73a11464e9cb7a8c38b6d2fcfbe` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:51.283418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8e9bb-b4c6-43db-8c22-7778c47a5672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=019ad73a11464e9cb7a8c38b6d2fcfbe)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:51.285465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8e9bb-b4c6-43db-8c22-7778c47a5672 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:51.458750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:51.505510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-60e8e9bb-b4c6-43db-8c22-7778c47a5672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '019ad73a11464e9cb7a8c38b6d2fcfbe'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.506162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-60e8e9bb-b4c6-43db-8c22-7778c47a5672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '019ad73a11464e9cb7a8c38b6d2fcfbe'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.513361 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-60e8e9bb-b4c6-43db-8c22-7778c47a5672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '019ad73a11464e9cb7a8c38b6d2fcfbe'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.562243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:51.564224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:51.595055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-60e8e9bb-b4c6-43db-8c22-7778c47a5672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '019ad73a11464e9cb7a8c38b6d2fcfbe'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.595763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-60e8e9bb-b4c6-43db-8c22-7778c47a5672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '019ad73a11464e9cb7a8c38b6d2fcfbe'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.619163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 961/1943] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:51 2022] DELETE /identity/v3/users/019ad73a11464e9cb7a8c38b6d2fcfbe => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:51.645329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29f77f-a03a-4e07-b7a8-70e4d97096aa admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:51.721716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a4ba0-ddd5-4061-bbce-d5ad825505b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:51.722376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a4ba0-ddd5-4061-bbce-d5ad825505b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:51.722661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a4ba0-ddd5-4061-bbce-d5ad825505b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:51.722940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a4ba0-ddd5-4061-bbce-d5ad825505b7 admin admin] PATH_INFO: `/v3/projects/b2a040ee218747f4a46046e948e9aa78` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:51.725296 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a4ba0-ddd5-4061-bbce-d5ad825505b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a040ee218747f4a46046e948e9aa78)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:51.726286 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a4ba0-ddd5-4061-bbce-d5ad825505b7 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:51.881558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-867a4ba0-ddd5-4061-bbce-d5ad825505b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a040ee218747f4a46046e948e9aa78', 'request_id': 'req-867a4ba0-ddd5-4061-bbce-d5ad825505b7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.891880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-867a4ba0-ddd5-4061-bbce-d5ad825505b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a040ee218747f4a46046e948e9aa78', 'request_id': 'req-867a4ba0-ddd5-4061-bbce-d5ad825505b7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:51.899077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-867a4ba0-ddd5-4061-bbce-d5ad825505b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a040ee218747f4a46046e948e9aa78', 'request_id': 'req-867a4ba0-ddd5-4061-bbce-d5ad825505b7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:52.055785 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-867a4ba0-ddd5-4061-bbce-d5ad825505b7 admin admin] The token cache is being invalidate because project b2a040ee218747f4a46046e948e9aa78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:52.056606 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-867a4ba0-ddd5-4061-bbce-d5ad825505b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:52.056881 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-867a4ba0-ddd5-4061-bbce-d5ad825505b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:52.059858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 983/1944] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:51 2022] DELETE /identity/v3/projects/b2a040ee218747f4a46046e948e9aa78 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:52.065827 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7367d7f-3a51-4cbf-94ca-6b075a5b4f0b None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:52.066222 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7367d7f-3a51-4cbf-94ca-6b075a5b4f0b None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:52.066557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7367d7f-3a51-4cbf-94ca-6b075a5b4f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:52.090645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-e7367d7f-3a51-4cbf-94ca-6b075a5b4f0b None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:52.091849 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e7367d7f-3a51-4cbf-94ca-6b075a5b4f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:52.149510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 962/1945] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:52.158201 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadef2cd-7f9e-40d7-9faf-24a21a8112af None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:52.158721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadef2cd-7f9e-40d7-9faf-24a21a8112af None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:52.158796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadef2cd-7f9e-40d7-9faf-24a21a8112af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:52.200797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-eadef2cd-7f9e-40d7-9faf-24a21a8112af None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:52.202173 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-eadef2cd-7f9e-40d7-9faf-24a21a8112af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:52.212452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 984/1946] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:52.217013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7367d7f-3a51-4cbf-94ca-6b075a5b4f0b None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:52.227276 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7367d7f-3a51-4cbf-94ca-6b075a5b4f0b None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:52.232238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603aab4a-c13d-40de-914b-727a2036336f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:52.232845 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603aab4a-c13d-40de-914b-727a2036336f admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:52.233125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603aab4a-c13d-40de-914b-727a2036336f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:52.233442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603aab4a-c13d-40de-914b-727a2036336f admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:52.234093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603aab4a-c13d-40de-914b-727a2036336f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:52.234397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603aab4a-c13d-40de-914b-727a2036336f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:52.236273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603aab4a-c13d-40de-914b-727a2036336f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:52.249380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 963/1947] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:52.253907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadef2cd-7f9e-40d7-9faf-24a21a8112af None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:52.263685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadef2cd-7f9e-40d7-9faf-24a21a8112af None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:52.268863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6edc7a-c88c-4401-b89f-39ed5f1148cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:52.269460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6edc7a-c88c-4401-b89f-39ed5f1148cf admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:52.269742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6edc7a-c88c-4401-b89f-39ed5f1148cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:52.270010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6edc7a-c88c-4401-b89f-39ed5f1148cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:52.270650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6edc7a-c88c-4401-b89f-39ed5f1148cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:52.272572 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6edc7a-c88c-4401-b89f-39ed5f1148cf admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:52.361895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 985/1948] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:52 2022] POST /identity/v3/projects => generated 357 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:52.366442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603aab4a-c13d-40de-914b-727a2036336f admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:52.377864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603aab4a-c13d-40de-914b-727a2036336f admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:52.408751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6edc7a-c88c-4401-b89f-39ed5f1148cf admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:52.412197 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2e6edc7a-c88c-4401-b89f-39ed5f1148cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:52.441846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cb1c7-b6dd-40ff-a10c-df1c1a1573fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:52.442543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cb1c7-b6dd-40ff-a10c-df1c1a1573fc admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:52.442774 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cb1c7-b6dd-40ff-a10c-df1c1a1573fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:52.443055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cb1c7-b6dd-40ff-a10c-df1c1a1573fc admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:52.443700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051cb1c7-b6dd-40ff-a10c-df1c1a1573fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:52.445701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051cb1c7-b6dd-40ff-a10c-df1c1a1573fc admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:52.481730 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6edc7a-c88c-4401-b89f-39ed5f1148cf admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:52.491499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:52.491499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:52.491499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:52.491499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb admin admin] PATH_INFO: `/v3/projects/6d60ed8b04ca46cc8b67d4c55af57b6d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:52.491891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d60ed8b04ca46cc8b67d4c55af57b6d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:52.491891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:52.520336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:52.561273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 964/1949] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:52 2022] POST /identity/v3/users => generated 279 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:52.565471 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cb1c7-b6dd-40ff-a10c-df1c1a1573fc admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:52.575288 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cb1c7-b6dd-40ff-a10c-df1c1a1573fc admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:52.580003 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87fe8a-8cc9-49a5-a1c7-631094b2cff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:52.580627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87fe8a-8cc9-49a5-a1c7-631094b2cff6 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:52.580974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87fe8a-8cc9-49a5-a1c7-631094b2cff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:52.581249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87fe8a-8cc9-49a5-a1c7-631094b2cff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:52.581880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87fe8a-8cc9-49a5-a1c7-631094b2cff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:52.583519 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87fe8a-8cc9-49a5-a1c7-631094b2cff6 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:52.595073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 965/1950] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:27:52.603412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87fe8a-8cc9-49a5-a1c7-631094b2cff6 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:52.607742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d60ed8b04ca46cc8b67d4c55af57b6d', 'request_id': 'req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:52.614444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87fe8a-8cc9-49a5-a1c7-631094b2cff6 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:52.620401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d60ed8b04ca46cc8b67d4c55af57b6d', 'request_id': 'req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:52.626440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d60ed8b04ca46cc8b67d4c55af57b6d', 'request_id': 'req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:52.670779 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43276eb-8a49-44c1-9dcd-0a1854770923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:52.671387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43276eb-8a49-44c1-9dcd-0a1854770923 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:52.671665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43276eb-8a49-44c1-9dcd-0a1854770923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:52.671934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43276eb-8a49-44c1-9dcd-0a1854770923 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:52.672637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43276eb-8a49-44c1-9dcd-0a1854770923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:52.674228 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43276eb-8a49-44c1-9dcd-0a1854770923 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:52.684483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 966/1951] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:27:52.688552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43276eb-8a49-44c1-9dcd-0a1854770923 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:52.702269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43276eb-8a49-44c1-9dcd-0a1854770923 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:52.702842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db749f-d50f-45e9-97f1-ba6c52e4d4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:52.703152 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db749f-d50f-45e9-97f1-ba6c52e4d4f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:52.703434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db749f-d50f-45e9-97f1-ba6c52e4d4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:52.703712 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db749f-d50f-45e9-97f1-ba6c52e4d4f1 admin admin] PATH_INFO: `/v3/projects/ed125e23c75a46c492967edecb146325/users/ebb3389c18c048008f7cfd118cc975ce/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:52.718113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db749f-d50f-45e9-97f1-ba6c52e4d4f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed125e23c75a46c492967edecb146325, user_id=ebb3389c18c048008f7cfd118cc975ce, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:52.722234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db749f-d50f-45e9-97f1-ba6c52e4d4f1 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:52.754334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb admin admin] The token cache is being invalidate because project 6d60ed8b04ca46cc8b67d4c55af57b6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:52.754635 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:52.756032 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f0ad408f-590c-4c5e-b30d-f9ee2f6b36bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:52.756819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 986/1952] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:52 2022] DELETE /identity/v3/projects/6d60ed8b04ca46cc8b67d4c55af57b6d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:52.761597 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0002ddb-4f14-4232-902c-d9115c656a64 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:52.761889 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0002ddb-4f14-4232-902c-d9115c656a64 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:52.762166 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0002ddb-4f14-4232-902c-d9115c656a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:52.783744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b0002ddb-4f14-4232-902c-d9115c656a64 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:52.784980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b0002ddb-4f14-4232-902c-d9115c656a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:52.794404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 987/1953] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:52.802549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bf2b7-53d1-4242-82d1-3aa8032bc575 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:52.802841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bf2b7-53d1-4242-82d1-3aa8032bc575 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:52.803124 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bf2b7-53d1-4242-82d1-3aa8032bc575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:52.812043 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 967/1954] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:52 2022] PUT /identity/v3/projects/ed125e23c75a46c492967edecb146325/users/ebb3389c18c048008f7cfd118cc975ce/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:52.824389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69732a89-95e5-4d78-aa42-265fe5032180 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:52.824659 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69732a89-95e5-4d78-aa42-265fe5032180 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:52.824973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69732a89-95e5-4d78-aa42-265fe5032180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:52.843859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ce8bf2b7-53d1-4242-82d1-3aa8032bc575 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:52.845196 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ce8bf2b7-53d1-4242-82d1-3aa8032bc575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:52.871211 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-69732a89-95e5-4d78-aa42-265fe5032180 None None] MFA Rules not processed for user `ebb3389c18c048008f7cfd118cc975ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:52.872470 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-69732a89-95e5-4d78-aa42-265fe5032180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:52.925291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 988/1955] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:52.930096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8bf2b7-53d1-4242-82d1-3aa8032bc575 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:52.939478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8bf2b7-53d1-4242-82d1-3aa8032bc575 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:52.944675 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4fdc2-7a1a-4055-8a90-96f04f6d0c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:52.945291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4fdc2-7a1a-4055-8a90-96f04f6d0c92 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:52.945621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4fdc2-7a1a-4055-8a90-96f04f6d0c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:52.945891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4fdc2-7a1a-4055-8a90-96f04f6d0c92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:52.946681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4fdc2-7a1a-4055-8a90-96f04f6d0c92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:52.947050 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4fdc2-7a1a-4055-8a90-96f04f6d0c92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:52.948696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4fdc2-7a1a-4055-8a90-96f04f6d0c92 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:52.962427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 989/1956] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:52.962566 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 968/1957] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:52.967078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69732a89-95e5-4d78-aa42-265fe5032180 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:53.004008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69732a89-95e5-4d78-aa42-265fe5032180 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:53.009927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d34060-bcb3-4f96-b3f8-7d2b8fded1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:53.010063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4fdc2-7a1a-4055-8a90-96f04f6d0c92 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:53.010746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d34060-bcb3-4f96-b3f8-7d2b8fded1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:53.011190 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d34060-bcb3-4f96-b3f8-7d2b8fded1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:53.011563 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d34060-bcb3-4f96-b3f8-7d2b8fded1f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:53.011765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-04b4fdc2-7a1a-4055-8a90-96f04f6d0c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:53.012488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d34060-bcb3-4f96-b3f8-7d2b8fded1f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:53.016322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d34060-bcb3-4f96-b3f8-7d2b8fded1f4 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:53.144849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4fdc2-7a1a-4055-8a90-96f04f6d0c92 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:53.151728 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32cdda-f416-41ff-b182-3a75a893c340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:53.153654 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32cdda-f416-41ff-b182-3a75a893c340 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:53.154094 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32cdda-f416-41ff-b182-3a75a893c340 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:53.154519 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32cdda-f416-41ff-b182-3a75a893c340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:53.160976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 969/1958] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:52 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:27:53.167628 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32cdda-f416-41ff-b182-3a75a893c340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:53.170255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32cdda-f416-41ff-b182-3a75a893c340 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:53.171970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d34060-bcb3-4f96-b3f8-7d2b8fded1f4 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:53.187249 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 990/1959] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:53.204944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32cdda-f416-41ff-b182-3a75a893c340 service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:53.206337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-0b32cdda-f416-41ff-b182-3a75a893c340 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:53.236862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32cdda-f416-41ff-b182-3a75a893c340 service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:53.284651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d34060-bcb3-4f96-b3f8-7d2b8fded1f4 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:53.304675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ce981-8022-4f28-aeb4-6c646c49d8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:53.305374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ce981-8022-4f28-aeb4-6c646c49d8ca admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:53.305723 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ce981-8022-4f28-aeb4-6c646c49d8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:53.306065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ce981-8022-4f28-aeb4-6c646c49d8ca admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:53.307305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ce981-8022-4f28-aeb4-6c646c49d8ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:53.310162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ce981-8022-4f28-aeb4-6c646c49d8ca admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:53.330611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4b6f8-6d90-481c-9917-50e7f4133d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:53.331289 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4b6f8-6d90-481c-9917-50e7f4133d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:53.332521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4b6f8-6d90-481c-9917-50e7f4133d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:53.332521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4b6f8-6d90-481c-9917-50e7f4133d9a admin admin] PATH_INFO: `/v3/users/ebb3389c18c048008f7cfd118cc975ce` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:53.333163 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4b6f8-6d90-481c-9917-50e7f4133d9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebb3389c18c048008f7cfd118cc975ce)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:53.335684 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4b6f8-6d90-481c-9917-50e7f4133d9a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:53.430886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 970/1960] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:53 2022] POST /identity/v3/users => generated 291 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:53.435438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ce981-8022-4f28-aeb4-6c646c49d8ca admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:53.455164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ce981-8022-4f28-aeb4-6c646c49d8ca admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:53.476956 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd30b6-a893-42ed-af5a-91d1a12229e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:53.477716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd30b6-a893-42ed-af5a-91d1a12229e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:53.478126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd30b6-a893-42ed-af5a-91d1a12229e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:53.478765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd30b6-a893-42ed-af5a-91d1a12229e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:53.480089 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd30b6-a893-42ed-af5a-91d1a12229e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:53.484011 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd30b6-a893-42ed-af5a-91d1a12229e9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:53.514521 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 971/1961] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:27:53.518680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd30b6-a893-42ed-af5a-91d1a12229e9 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:53.533647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd30b6-a893-42ed-af5a-91d1a12229e9 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:53.551345 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70b2d6-756c-4966-9660-0a1d4b5eb2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:53.552158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70b2d6-756c-4966-9660-0a1d4b5eb2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:53.552949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70b2d6-756c-4966-9660-0a1d4b5eb2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:53.552949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70b2d6-756c-4966-9660-0a1d4b5eb2a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:53.553671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70b2d6-756c-4966-9660-0a1d4b5eb2a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:53.555337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70b2d6-756c-4966-9660-0a1d4b5eb2a8 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:53.561051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:53.565988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a3d4b6f8-6d90-481c-9917-50e7f4133d9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebb3389c18c048008f7cfd118cc975ce'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:53.575355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a3d4b6f8-6d90-481c-9917-50e7f4133d9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebb3389c18c048008f7cfd118cc975ce'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:53.578987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 972/1962] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:53.580114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70b2d6-756c-4966-9660-0a1d4b5eb2a8 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:53.593054 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70b2d6-756c-4966-9660-0a1d4b5eb2a8 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:53.600650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74acf63-b364-42c7-b7d5-ded865a892fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:53.602850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74acf63-b364-42c7-b7d5-ded865a892fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:53.603279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74acf63-b364-42c7-b7d5-ded865a892fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:53.603912 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74acf63-b364-42c7-b7d5-ded865a892fb admin admin] PATH_INFO: `/v3/projects/c71576f2f02a4392848a14bde478e145/users/72fd224707df443e942910892293efbc/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:53.625850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74acf63-b364-42c7-b7d5-ded865a892fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c71576f2f02a4392848a14bde478e145, user_id=72fd224707df443e942910892293efbc, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:53.629325 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74acf63-b364-42c7-b7d5-ded865a892fb admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:53.664515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a3d4b6f8-6d90-481c-9917-50e7f4133d9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebb3389c18c048008f7cfd118cc975ce'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:53.665140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a3d4b6f8-6d90-481c-9917-50e7f4133d9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebb3389c18c048008f7cfd118cc975ce'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:53.678751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a3d4b6f8-6d90-481c-9917-50e7f4133d9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebb3389c18c048008f7cfd118cc975ce'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:53.685440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 991/1963] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:53 2022] DELETE /identity/v3/users/ebb3389c18c048008f7cfd118cc975ce => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:53.690033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acc9c2-1ef9-4ed4-bbba-1264d7db58a2 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:53.690321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acc9c2-1ef9-4ed4-bbba-1264d7db58a2 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:53.690649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acc9c2-1ef9-4ed4-bbba-1264d7db58a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:53.709780 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-c1acc9c2-1ef9-4ed4-bbba-1264d7db58a2 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:53.710994 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c1acc9c2-1ef9-4ed4-bbba-1264d7db58a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:53.767772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 973/1964] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:53 2022] PUT /identity/v3/projects/c71576f2f02a4392848a14bde478e145/users/72fd224707df443e942910892293efbc/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:53.770458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 992/1965] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:53.778422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43344bf9-8ec4-4bfc-acf6-05bc8dfb41b7 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:53.778715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43344bf9-8ec4-4bfc-acf6-05bc8dfb41b7 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:53.778994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43344bf9-8ec4-4bfc-acf6-05bc8dfb41b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:53.780569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acc9c2-1ef9-4ed4-bbba-1264d7db58a2 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:53.781941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c1acc9c2-1ef9-4ed4-bbba-1264d7db58a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:53.807685 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-43344bf9-8ec4-4bfc-acf6-05bc8dfb41b7 None None] MFA Rules not processed for user `72fd224707df443e942910892293efbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:53.808931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-43344bf9-8ec4-4bfc-acf6-05bc8dfb41b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:53.840455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acc9c2-1ef9-4ed4-bbba-1264d7db58a2 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:53.880057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 974/1966] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:53.884638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43344bf9-8ec4-4bfc-acf6-05bc8dfb41b7 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:53.886183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-43344bf9-8ec4-4bfc-acf6-05bc8dfb41b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:53.890042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977302d-1014-4420-b354-71ab33215947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:53.890669 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977302d-1014-4420-b354-71ab33215947 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:53.890949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977302d-1014-4420-b354-71ab33215947 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:53.891225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977302d-1014-4420-b354-71ab33215947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:53.957028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977302d-1014-4420-b354-71ab33215947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:53.957666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43344bf9-8ec4-4bfc-acf6-05bc8dfb41b7 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:53.958670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977302d-1014-4420-b354-71ab33215947 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:53.962425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8870c5-557f-40bb-b235-3b9510695eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:53.963039 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8870c5-557f-40bb-b235-3b9510695eae admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:53.963324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8870c5-557f-40bb-b235-3b9510695eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:53.963601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8870c5-557f-40bb-b235-3b9510695eae admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:53.964263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8870c5-557f-40bb-b235-3b9510695eae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:53.966980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8870c5-557f-40bb-b235-3b9510695eae admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:53.971157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 993/1967] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:27:54.156058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 975/1968] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:53 2022] POST /identity/v3/projects => generated 383 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:54.160187 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977302d-1014-4420-b354-71ab33215947 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:54.170075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977302d-1014-4420-b354-71ab33215947 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:54.223657 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:54.225870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e61b1-afc0-4510-a5da-ce4f3c8bb5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:54.226480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e61b1-afc0-4510-a5da-ce4f3c8bb5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:54.226761 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e61b1-afc0-4510-a5da-ce4f3c8bb5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:54.227034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e61b1-afc0-4510-a5da-ce4f3c8bb5a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:54.227696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e61b1-afc0-4510-a5da-ce4f3c8bb5a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:54.229659 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e61b1-afc0-4510-a5da-ce4f3c8bb5a2 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:54.383375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 994/1969] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:54 2022] POST /identity/v3/users => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:54.388364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8870c5-557f-40bb-b235-3b9510695eae admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:54.399383 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8870c5-557f-40bb-b235-3b9510695eae admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:54.404713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3b5a7-4f79-4417-9579-c599bbc800b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:54.405324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3b5a7-4f79-4417-9579-c599bbc800b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:54.405607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3b5a7-4f79-4417-9579-c599bbc800b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:54.405880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3b5a7-4f79-4417-9579-c599bbc800b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:54.406526 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3b5a7-4f79-4417-9579-c599bbc800b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:54.408209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3b5a7-4f79-4417-9579-c599bbc800b0 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:54.420202 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 976/1970] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:54.428720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e61b1-afc0-4510-a5da-ce4f3c8bb5a2 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:54.443177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e61b1-afc0-4510-a5da-ce4f3c8bb5a2 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:54.448319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322818a5-81fb-4b9d-99cf-b7d9077b286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:54.449806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322818a5-81fb-4b9d-99cf-b7d9077b286f admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:54.450190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322818a5-81fb-4b9d-99cf-b7d9077b286f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:54.450587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322818a5-81fb-4b9d-99cf-b7d9077b286f admin admin] PATH_INFO: `/v3/projects/df732d28d4bb492a972eb7623268fd76/users/5988292c3335465e8c253edc0474680e/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:54.467328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322818a5-81fb-4b9d-99cf-b7d9077b286f admin admin] RBAC: Authorizing `identity:create_grant(project_id=df732d28d4bb492a972eb7623268fd76, user_id=5988292c3335465e8c253edc0474680e, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:54.470094 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322818a5-81fb-4b9d-99cf-b7d9077b286f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:54.563393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3b5a7-4f79-4417-9579-c599bbc800b0 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:54.564116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 995/1971] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:54 2022] PUT /identity/v3/projects/df732d28d4bb492a972eb7623268fd76/users/5988292c3335465e8c253edc0474680e/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:54.565489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-82b3b5a7-4f79-4417-9579-c599bbc800b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:54.570573 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322818a5-81fb-4b9d-99cf-b7d9077b286f admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:54.594739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3b5a7-4f79-4417-9579-c599bbc800b0 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:54.667170 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322818a5-81fb-4b9d-99cf-b7d9077b286f admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:54.673246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a389e7-18d7-4485-ab30-38262f32f2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:54.674131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a389e7-18d7-4485-ab30-38262f32f2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:54.674522 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a389e7-18d7-4485-ab30-38262f32f2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:54.674898 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a389e7-18d7-4485-ab30-38262f32f2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:54.676581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a389e7-18d7-4485-ab30-38262f32f2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:54.681321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a389e7-18d7-4485-ab30-38262f32f2b0 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:54.683225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:54.685090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:54.685572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:54.686547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc admin admin] PATH_INFO: `/v3/projects/ed125e23c75a46c492967edecb146325` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:54.688045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed125e23c75a46c492967edecb146325)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:54.690160 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:54.695415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 996/1972] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:54.706068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a389e7-18d7-4485-ab30-38262f32f2b0 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:54.727125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a389e7-18d7-4485-ab30-38262f32f2b0 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:54.736266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a7c88-72b9-49bf-a5f0-828f44ccf024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:54.736266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a7c88-72b9-49bf-a5f0-828f44ccf024 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:54.736266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a7c88-72b9-49bf-a5f0-828f44ccf024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:54.736266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a7c88-72b9-49bf-a5f0-828f44ccf024 admin admin] PATH_INFO: `/v3/projects/df732d28d4bb492a972eb7623268fd76/users/5988292c3335465e8c253edc0474680e/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:54.737002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a7c88-72b9-49bf-a5f0-828f44ccf024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df732d28d4bb492a972eb7623268fd76, user_id=5988292c3335465e8c253edc0474680e, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:54.739758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a7c88-72b9-49bf-a5f0-828f44ccf024 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:54.751462 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 997/1973] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:54 2022] PUT /identity/v3/projects/df732d28d4bb492a972eb7623268fd76/users/5988292c3335465e8c253edc0474680e/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:54.762333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdd25c-ebe8-40b3-9a43-e2afd78d0d6e None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:54.762621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdd25c-ebe8-40b3-9a43-e2afd78d0d6e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:54.762899 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdd25c-ebe8-40b3-9a43-e2afd78d0d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:54.790691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-14fdd25c-ebe8-40b3-9a43-e2afd78d0d6e None None] MFA Rules not processed for user `5988292c3335465e8c253edc0474680e`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:54.791918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-14fdd25c-ebe8-40b3-9a43-e2afd78d0d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:54.864968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 998/1974] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:54.876201 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdd25c-ebe8-40b3-9a43-e2afd78d0d6e None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:54.877545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-14fdd25c-ebe8-40b3-9a43-e2afd78d0d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:54.933443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdd25c-ebe8-40b3-9a43-e2afd78d0d6e None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:54.937771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3865ba6-2770-4df5-8dc8-ab448ac3fc87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:54.938370 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3865ba6-2770-4df5-8dc8-ab448ac3fc87 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:54.938646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3865ba6-2770-4df5-8dc8-ab448ac3fc87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:54.938915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3865ba6-2770-4df5-8dc8-ab448ac3fc87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:54.948066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3865ba6-2770-4df5-8dc8-ab448ac3fc87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:54.949556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3865ba6-2770-4df5-8dc8-ab448ac3fc87 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:54.954518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed125e23c75a46c492967edecb146325', 'request_id': 'req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:54.959641 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 999/1975] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:27:54.961923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed125e23c75a46c492967edecb146325', 'request_id': 'req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:55.074725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed125e23c75a46c492967edecb146325', 'request_id': 'req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:55.085951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc admin admin] The token cache is being invalidate because project ed125e23c75a46c492967edecb146325 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:55.086243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:55.086868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c6a30972-8487-4ad0-86d7-9eb47d39c2dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:55.088387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 977/1976] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:54 2022] DELETE /identity/v3/projects/ed125e23c75a46c492967edecb146325 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:55.093773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5672c2-b38d-4c10-bd18-37d857d5fcd9 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:55.094062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5672c2-b38d-4c10-bd18-37d857d5fcd9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:55.094367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5672c2-b38d-4c10-bd18-37d857d5fcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:55.114227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3e5672c2-b38d-4c10-bd18-37d857d5fcd9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:55.115420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3e5672c2-b38d-4c10-bd18-37d857d5fcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:55.185396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1000/1977] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:55.193609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d3d90-8312-4ca0-96ff-ddc2ac7220fd None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:55.194002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d3d90-8312-4ca0-96ff-ddc2ac7220fd None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:55.194283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d3d90-8312-4ca0-96ff-ddc2ac7220fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:55.215559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-ee9d3d90-8312-4ca0-96ff-ddc2ac7220fd None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:55.216843 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ee9d3d90-8312-4ca0-96ff-ddc2ac7220fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:55.227230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 978/1978] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:55.229130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5672c2-b38d-4c10-bd18-37d857d5fcd9 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:55.240055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5672c2-b38d-4c10-bd18-37d857d5fcd9 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:55.245020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5e2f2-9cea-4fa8-8389-e09c9cebe2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:55.245730 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5e2f2-9cea-4fa8-8389-e09c9cebe2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:55.246106 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5e2f2-9cea-4fa8-8389-e09c9cebe2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:55.246599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5e2f2-9cea-4fa8-8389-e09c9cebe2a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:55.247168 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5e2f2-9cea-4fa8-8389-e09c9cebe2a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:27:55.247566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5e2f2-9cea-4fa8-8389-e09c9cebe2a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:55.249455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5e2f2-9cea-4fa8-8389-e09c9cebe2a0 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:55.261779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1001/1979] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:27:55.267095 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d3d90-8312-4ca0-96ff-ddc2ac7220fd None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:55.277086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d3d90-8312-4ca0-96ff-ddc2ac7220fd None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:55.282502 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375745d-cb82-48bd-a0ad-319fbcec3656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:55.283168 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375745d-cb82-48bd-a0ad-319fbcec3656 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:55.283448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375745d-cb82-48bd-a0ad-319fbcec3656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:55.283720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375745d-cb82-48bd-a0ad-319fbcec3656 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:55.284397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375745d-cb82-48bd-a0ad-319fbcec3656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:55.286532 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375745d-cb82-48bd-a0ad-319fbcec3656 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:55.373367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5e2f2-9cea-4fa8-8389-e09c9cebe2a0 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:55.375167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b5f5e2f2-9cea-4fa8-8389-e09c9cebe2a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:55.396845 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 979/1980] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:27:55 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:27:55.400910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375745d-cb82-48bd-a0ad-319fbcec3656 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:55.436943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5e2f2-9cea-4fa8-8389-e09c9cebe2a0 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:55.464735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375745d-cb82-48bd-a0ad-319fbcec3656 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:55.470523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c5e27-da35-4807-81db-6d139d8c1749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:55.471142 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c5e27-da35-4807-81db-6d139d8c1749 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:55.471426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c5e27-da35-4807-81db-6d139d8c1749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:55.471703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c5e27-da35-4807-81db-6d139d8c1749 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:55.472395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c5e27-da35-4807-81db-6d139d8c1749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:55.474729 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c5e27-da35-4807-81db-6d139d8c1749 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:55.488016 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890d092-a781-42a7-9178-fb5e77bc04ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:55.488723 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890d092-a781-42a7-9178-fb5e77bc04ad service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:55.489016 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890d092-a781-42a7-9178-fb5e77bc04ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:55.494371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890d092-a781-42a7-9178-fb5e77bc04ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:55.494371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8890d092-a781-42a7-9178-fb5e77bc04ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:55.571341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890d092-a781-42a7-9178-fb5e77bc04ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:55.572701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890d092-a781-42a7-9178-fb5e77bc04ad service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:55.585767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1002/1981] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:55.642045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 980/1982] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:27:55 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:27:55.647052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890d092-a781-42a7-9178-fb5e77bc04ad service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:55.660768 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890d092-a781-42a7-9178-fb5e77bc04ad service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:55.666268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61c095-061a-4bb3-88a5-9de4cec147cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:55.666947 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61c095-061a-4bb3-88a5-9de4cec147cc admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:55.668238 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61c095-061a-4bb3-88a5-9de4cec147cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:55.671634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61c095-061a-4bb3-88a5-9de4cec147cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:55.682567 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61c095-061a-4bb3-88a5-9de4cec147cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:55.682567 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61c095-061a-4bb3-88a5-9de4cec147cc admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:55.696728 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1003/1983] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:55.697958 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c5e27-da35-4807-81db-6d139d8c1749 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:55.715515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c5e27-da35-4807-81db-6d139d8c1749 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:55.721710 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de75a7-0bb5-47a7-a209-56e33c765bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:55.722744 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de75a7-0bb5-47a7-a209-56e33c765bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:55.723367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de75a7-0bb5-47a7-a209-56e33c765bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:55.723840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de75a7-0bb5-47a7-a209-56e33c765bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:55.724750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de75a7-0bb5-47a7-a209-56e33c765bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:55.726689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de75a7-0bb5-47a7-a209-56e33c765bd9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:55.742394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 981/1984] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:55.745772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61c095-061a-4bb3-88a5-9de4cec147cc admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:55.760297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61c095-061a-4bb3-88a5-9de4cec147cc admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:55.767286 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e2c04-cc5c-44d1-888c-c51d1efe21fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:55.768878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e2c04-cc5c-44d1-888c-c51d1efe21fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:55.769165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e2c04-cc5c-44d1-888c-c51d1efe21fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:55.769445 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e2c04-cc5c-44d1-888c-c51d1efe21fc admin admin] PATH_INFO: `/v3/projects/c43ecc2fc17946c28bf1ceb32cd6198d/users/5eeab15ac1764986b11d0aa556e17d2f/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:55.785536 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e2c04-cc5c-44d1-888c-c51d1efe21fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c43ecc2fc17946c28bf1ceb32cd6198d, user_id=5eeab15ac1764986b11d0aa556e17d2f, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:55.788349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e2c04-cc5c-44d1-888c-c51d1efe21fc admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:55.915051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:55.920679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1004/1985] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:27:55 2022] PUT /identity/v3/projects/c43ecc2fc17946c28bf1ceb32cd6198d/users/5eeab15ac1764986b11d0aa556e17d2f/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:27:55.930955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b8fe9-88ba-4e2b-b3cd-41484f0c96a8 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:55.931305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b8fe9-88ba-4e2b-b3cd-41484f0c96a8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:55.931587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b8fe9-88ba-4e2b-b3cd-41484f0c96a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:55.959770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-d81b8fe9-88ba-4e2b-b3cd-41484f0c96a8 None None] MFA Rules not processed for user `5eeab15ac1764986b11d0aa556e17d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:55.961057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-d81b8fe9-88ba-4e2b-b3cd-41484f0c96a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:56.031124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 982/1986] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:56.177797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:56.316429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e2c04-cc5c-44d1-888c-c51d1efe21fc admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:56.318452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-711e2c04-cc5c-44d1-888c-c51d1efe21fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:56.336891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b8fe9-88ba-4e2b-b3cd-41484f0c96a8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:56.338290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-d81b8fe9-88ba-4e2b-b3cd-41484f0c96a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:56.386144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e2c04-cc5c-44d1-888c-c51d1efe21fc admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:56.391531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:56.392203 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:56.392866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:56.393156 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] PATH_INFO: `/v3/users/85b7791eb6164be59465e506e4edf559` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:56.394376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b7791eb6164be59465e506e4edf559)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:56.396374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:56.397592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b8fe9-88ba-4e2b-b3cd-41484f0c96a8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:56.402619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d3443-f927-4ede-817d-e0a6b63628f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:56.403165 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d3443-f927-4ede-817d-e0a6b63628f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:56.403443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d3443-f927-4ede-817d-e0a6b63628f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:56.403716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d3443-f927-4ede-817d-e0a6b63628f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:56.412446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d3443-f927-4ede-817d-e0a6b63628f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:56.414240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d3443-f927-4ede-817d-e0a6b63628f8 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:56.423117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 983/1987] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:56.653714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b7791eb6164be59465e506e4edf559'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:56.661032 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b7791eb6164be59465e506e4edf559'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:56.784215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b7791eb6164be59465e506e4edf559'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:56.785078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b7791eb6164be59465e506e4edf559'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:56.800960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b7791eb6164be59465e506e4edf559'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:56.808440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1005/1988] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:56 2022] DELETE /identity/v3/users/85b7791eb6164be59465e506e4edf559 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:56.813298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd201d24-e3a8-4809-9187-2c5b46b17372 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:56.814070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd201d24-e3a8-4809-9187-2c5b46b17372 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:56.814360 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd201d24-e3a8-4809-9187-2c5b46b17372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:56.837010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-cd201d24-e3a8-4809-9187-2c5b46b17372 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:56.838206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-cd201d24-e3a8-4809-9187-2c5b46b17372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:56.900407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 984/1989] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:27:56.912151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:56.913990 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:56.974876 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbb6c8-236e-4cef-8651-a2d26232e198 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:56.978648 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638eb909-214a-4c98-b8bc-8e1e82afb4d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:56.979334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638eb909-214a-4c98-b8bc-8e1e82afb4d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:56.979617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638eb909-214a-4c98-b8bc-8e1e82afb4d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:56.979893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638eb909-214a-4c98-b8bc-8e1e82afb4d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:56.990624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638eb909-214a-4c98-b8bc-8e1e82afb4d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:56.992078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638eb909-214a-4c98-b8bc-8e1e82afb4d3 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:57.002250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1006/1990] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:57.728112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd201d24-e3a8-4809-9187-2c5b46b17372 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:57.729705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-cd201d24-e3a8-4809-9187-2c5b46b17372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:57.743823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd201d24-e3a8-4809-9187-2c5b46b17372 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:57.750143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:57.750876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:57.751279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:57.751695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] PATH_INFO: `/v3/projects/c9fee0114516417f982510947e169cb2` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:57.753566 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9fee0114516417f982510947e169cb2)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:57.755498 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:57.974300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=81,keystone:INSERT=5,keystone:DELETE=14 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:58.005350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9fee0114516417f982510947e169cb2', 'request_id': 'req-06effe51-796c-448f-9a4e-6aebb6ad655f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:58.016666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9fee0114516417f982510947e169cb2', 'request_id': 'req-06effe51-796c-448f-9a4e-6aebb6ad655f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:58.183359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9fee0114516417f982510947e169cb2', 'request_id': 'req-06effe51-796c-448f-9a4e-6aebb6ad655f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:58.207821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] The token cache is being invalidate because project c9fee0114516417f982510947e169cb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:58.208230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:58.208966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:58.212044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 985/1991] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:57 2022] DELETE /identity/v3/projects/c9fee0114516417f982510947e169cb2 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:58.219835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638eb909-214a-4c98-b8bc-8e1e82afb4d3 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:58.229852 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638eb909-214a-4c98-b8bc-8e1e82afb4d3 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:58.304054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:58.304959 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:58.305689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:58.306250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] PATH_INFO: `/v3/users/0b661042e627417ea0910afd43ce8061` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:58.308442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b661042e627417ea0910afd43ce8061)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:58.310725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:58.471949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:58.619889 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b661042e627417ea0910afd43ce8061'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:58.627672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b661042e627417ea0910afd43ce8061'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:58.733152 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b661042e627417ea0910afd43ce8061'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:58.733855 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b661042e627417ea0910afd43ce8061'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:58.770172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b661042e627417ea0910afd43ce8061'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:58.780272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1007/1992] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:58 2022] DELETE /identity/v3/users/0b661042e627417ea0910afd43ce8061 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:59.476859 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:59.479611 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:59.503080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06effe51-796c-448f-9a4e-6aebb6ad655f admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:59.576203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:59.577070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:59.577513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:59.577961 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] PATH_INFO: `/v3/projects/9e1c5a8077f34ee59168a109baf082c1` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:59.579256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e1c5a8077f34ee59168a109baf082c1)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:59.581298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:59.739100 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e1c5a8077f34ee59168a109baf082c1', 'request_id': 'req-d5df9fae-068b-4e93-8930-1286255b5367'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:59.744896 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e1c5a8077f34ee59168a109baf082c1', 'request_id': 'req-d5df9fae-068b-4e93-8930-1286255b5367'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:59.829934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e1c5a8077f34ee59168a109baf082c1', 'request_id': 'req-d5df9fae-068b-4e93-8930-1286255b5367'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:59.851335 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] The token cache is being invalidate because project 9e1c5a8077f34ee59168a109baf082c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:27:59.851615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:59.852297 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:27:59.853816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 986/1993] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:27:59 2022] DELETE /identity/v3/projects/9e1c5a8077f34ee59168a109baf082c1 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:27:59.858686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:59.860268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:59.874108 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ac21e-38fb-4887-a9e3-95d76b4e10e3 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:59.935247 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:59.935927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:59.936239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:59.936524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] PATH_INFO: `/v3/users/09b9b5a046104932ae00e4be9cc9d69c` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:59.937918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b9b5a046104932ae00e4be9cc9d69c)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:59.939783 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:00.245372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:00.247232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:00.250845 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b9b5a046104932ae00e4be9cc9d69c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:00.258505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b9b5a046104932ae00e4be9cc9d69c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:00.267602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df9fae-068b-4e93-8930-1286255b5367 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:00.267602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:00.267602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:00.267602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:00.268275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] PATH_INFO: `/v3/users/72fd224707df443e942910892293efbc` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:00.269112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72fd224707df443e942910892293efbc)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:00.271109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:00.406292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b9b5a046104932ae00e4be9cc9d69c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:00.407095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b9b5a046104932ae00e4be9cc9d69c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:00.422869 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b9b5a046104932ae00e4be9cc9d69c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:00.429213 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1008/1994] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:27:59 2022] DELETE /identity/v3/users/09b9b5a046104932ae00e4be9cc9d69c => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:00.697158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72fd224707df443e942910892293efbc'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:00.697878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72fd224707df443e942910892293efbc'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:00.711966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72fd224707df443e942910892293efbc'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:00.733387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:00.735345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:00.804371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2561dae-eb2a-4bdc-be3c-685532ba7bcd admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:00.809746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86726d7e-9eab-4ac7-b910-ed11c457e940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:00.810423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86726d7e-9eab-4ac7-b910-ed11c457e940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:00.810704 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86726d7e-9eab-4ac7-b910-ed11c457e940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:00.811005 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86726d7e-9eab-4ac7-b910-ed11c457e940 admin admin] PATH_INFO: `/v3/users/e3a89ef321fc49e0ae114bdf8244925d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:00.812300 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86726d7e-9eab-4ac7-b910-ed11c457e940 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3a89ef321fc49e0ae114bdf8244925d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:00.815701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86726d7e-9eab-4ac7-b910-ed11c457e940 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:00.852006 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72fd224707df443e942910892293efbc'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:00.859053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72fd224707df443e942910892293efbc'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:00.870987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 987/1995] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:00 2022] DELETE /identity/v3/users/72fd224707df443e942910892293efbc => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:00.872456 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:00.873821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:01.098777 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bb332-6232-4b97-973d-f8ff3572c6c1 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:01.098777 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff3729-87d8-4adb-a1bb-19228d0e4522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:01.098777 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff3729-87d8-4adb-a1bb-19228d0e4522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:01.098777 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff3729-87d8-4adb-a1bb-19228d0e4522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:01.099270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff3729-87d8-4adb-a1bb-19228d0e4522 admin admin] PATH_INFO: `/v3/users/5eeab15ac1764986b11d0aa556e17d2f` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:01.099270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff3729-87d8-4adb-a1bb-19228d0e4522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eeab15ac1764986b11d0aa556e17d2f)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:01.099270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff3729-87d8-4adb-a1bb-19228d0e4522 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:01.476858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:DELETE=5,keystone:SELECT=95 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:01.579772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-86726d7e-9eab-4ac7-b910-ed11c457e940 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3a89ef321fc49e0ae114bdf8244925d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:01.587110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-86726d7e-9eab-4ac7-b910-ed11c457e940 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3a89ef321fc49e0ae114bdf8244925d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:01.623909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94ff3729-87d8-4adb-a1bb-19228d0e4522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eeab15ac1764986b11d0aa556e17d2f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:01.624476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94ff3729-87d8-4adb-a1bb-19228d0e4522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eeab15ac1764986b11d0aa556e17d2f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:01.632881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94ff3729-87d8-4adb-a1bb-19228d0e4522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eeab15ac1764986b11d0aa556e17d2f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:01.663550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-86726d7e-9eab-4ac7-b910-ed11c457e940 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3a89ef321fc49e0ae114bdf8244925d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:01.667263 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-86726d7e-9eab-4ac7-b910-ed11c457e940 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3a89ef321fc49e0ae114bdf8244925d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:01.678449 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-86726d7e-9eab-4ac7-b910-ed11c457e940 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3a89ef321fc49e0ae114bdf8244925d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:01.685325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1009/1996] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:00 2022] DELETE /identity/v3/users/e3a89ef321fc49e0ae114bdf8244925d => generated 0 bytes in 953 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:01.687158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348cc11-2f6d-416c-b1dd-93fa0b8a330a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:01.687449 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348cc11-2f6d-416c-b1dd-93fa0b8a330a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:01.687722 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348cc11-2f6d-416c-b1dd-93fa0b8a330a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:01.711947 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-9348cc11-2f6d-416c-b1dd-93fa0b8a330a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:01.713210 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9348cc11-2f6d-416c-b1dd-93fa0b8a330a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:01.772826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1010/1997] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:01.773682 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348cc11-2f6d-416c-b1dd-93fa0b8a330a None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:01.774952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9348cc11-2f6d-416c-b1dd-93fa0b8a330a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:01.783110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94ff3729-87d8-4adb-a1bb-19228d0e4522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eeab15ac1764986b11d0aa556e17d2f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:01.783762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94ff3729-87d8-4adb-a1bb-19228d0e4522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eeab15ac1764986b11d0aa556e17d2f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:01.785040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:01.789637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348cc11-2f6d-416c-b1dd-93fa0b8a330a None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:01.794039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce8928-bc97-4c1f-8414-46ad34ea4779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:01.794672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce8928-bc97-4c1f-8414-46ad34ea4779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:01.794948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce8928-bc97-4c1f-8414-46ad34ea4779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:01.795228 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce8928-bc97-4c1f-8414-46ad34ea4779 admin admin] PATH_INFO: `/v3/projects/3f228bd771994ec6887d316f90ae91f6` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:01.796437 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce8928-bc97-4c1f-8414-46ad34ea4779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f228bd771994ec6887d316f90ae91f6)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:01.798347 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce8928-bc97-4c1f-8414-46ad34ea4779 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:01.802728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 988/1998] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:00 2022] DELETE /identity/v3/users/5eeab15ac1764986b11d0aa556e17d2f => generated 0 bytes in 931 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:01.803951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c86f11-d0f0-4c6d-b5d4-1c7e486dd043 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:01.804257 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c86f11-d0f0-4c6d-b5d4-1c7e486dd043 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:01.804529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c86f11-d0f0-4c6d-b5d4-1c7e486dd043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:01.823818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-99c86f11-d0f0-4c6d-b5d4-1c7e486dd043 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:01.825671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-99c86f11-d0f0-4c6d-b5d4-1c7e486dd043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:01.882562 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 989/1999] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:01.883385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c86f11-d0f0-4c6d-b5d4-1c7e486dd043 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:01.884666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-99c86f11-d0f0-4c6d-b5d4-1c7e486dd043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:01.926806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-75ce8928-bc97-4c1f-8414-46ad34ea4779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f228bd771994ec6887d316f90ae91f6', 'request_id': 'req-75ce8928-bc97-4c1f-8414-46ad34ea4779'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:01.932893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-75ce8928-bc97-4c1f-8414-46ad34ea4779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f228bd771994ec6887d316f90ae91f6', 'request_id': 'req-75ce8928-bc97-4c1f-8414-46ad34ea4779'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:01.938772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-75ce8928-bc97-4c1f-8414-46ad34ea4779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f228bd771994ec6887d316f90ae91f6', 'request_id': 'req-75ce8928-bc97-4c1f-8414-46ad34ea4779'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:01.942590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c86f11-d0f0-4c6d-b5d4-1c7e486dd043 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:01.991246 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01328a31-5e66-4905-a706-0d75e306f6a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:01.991781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01328a31-5e66-4905-a706-0d75e306f6a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:01.992064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01328a31-5e66-4905-a706-0d75e306f6a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:01.992368 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01328a31-5e66-4905-a706-0d75e306f6a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:01.994244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-01328a31-5e66-4905-a706-0d75e306f6a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:02.051144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-75ce8928-bc97-4c1f-8414-46ad34ea4779 admin admin] The token cache is being invalidate because project 3f228bd771994ec6887d316f90ae91f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:02.051458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-75ce8928-bc97-4c1f-8414-46ad34ea4779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:02.052116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-75ce8928-bc97-4c1f-8414-46ad34ea4779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:02.053672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1011/2000] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:01 2022] DELETE /identity/v3/projects/3f228bd771994ec6887d316f90ae91f6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:02.054899 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da620b9a-fbbe-44fc-9c9d-472c64d1d62e None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.055184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da620b9a-fbbe-44fc-9c9d-472c64d1d62e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.055465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da620b9a-fbbe-44fc-9c9d-472c64d1d62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.056495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01328a31-5e66-4905-a706-0d75e306f6a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:02.058027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01328a31-5e66-4905-a706-0d75e306f6a2 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:02.069461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 990/2001] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:02.070180 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01328a31-5e66-4905-a706-0d75e306f6a2 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:02.075714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-da620b9a-fbbe-44fc-9c9d-472c64d1d62e None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:02.077035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-da620b9a-fbbe-44fc-9c9d-472c64d1d62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:02.078988 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01328a31-5e66-4905-a706-0d75e306f6a2 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:02.082865 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e36f1-b621-47f5-a86a-ad5477683f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:02.083465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e36f1-b621-47f5-a86a-ad5477683f64 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.083747 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e36f1-b621-47f5-a86a-ad5477683f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.084023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e36f1-b621-47f5-a86a-ad5477683f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.085876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-555e36f1-b621-47f5-a86a-ad5477683f64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:02.089703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1012/2002] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:02.089703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894eae1-f889-45c8-8460-e776ac373016 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.089703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894eae1-f889-45c8-8460-e776ac373016 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.089703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894eae1-f889-45c8-8460-e776ac373016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.100810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e36f1-b621-47f5-a86a-ad5477683f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:02.102317 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e36f1-b621-47f5-a86a-ad5477683f64 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:02.111264 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b894eae1-f889-45c8-8460-e776ac373016 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:02.112955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b894eae1-f889-45c8-8460-e776ac373016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:02.114633 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 991/2003] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:02.115826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e36f1-b621-47f5-a86a-ad5477683f64 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:02.117719 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-555e36f1-b621-47f5-a86a-ad5477683f64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:02.123691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1013/2004] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:02.132370 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ec045-315c-49cc-aae5-018048a5a1e9 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.132718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ec045-315c-49cc-aae5-018048a5a1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.133038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ec045-315c-49cc-aae5-018048a5a1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.140881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e36f1-b621-47f5-a86a-ad5477683f64 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:02.146126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b98d8-0670-416c-82b1-01814de35343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:02.147177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b98d8-0670-416c-82b1-01814de35343 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.147177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b98d8-0670-416c-82b1-01814de35343 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.147392 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b98d8-0670-416c-82b1-01814de35343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.154862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-7e7ec045-315c-49cc-aae5-018048a5a1e9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:02.161257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7e7ec045-315c-49cc-aae5-018048a5a1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:02.161395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b98d8-0670-416c-82b1-01814de35343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:02.161395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b98d8-0670-416c-82b1-01814de35343 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:02.171384 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1014/2005] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:28:02.171554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 992/2006] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:02.171938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ec045-315c-49cc-aae5-018048a5a1e9 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:02.181126 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ec045-315c-49cc-aae5-018048a5a1e9 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:02.186836 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01a107-d861-4467-bf8b-5e5ff18259f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:02.187662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01a107-d861-4467-bf8b-5e5ff18259f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.188092 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01a107-d861-4467-bf8b-5e5ff18259f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.188552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01a107-d861-4467-bf8b-5e5ff18259f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.189376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01a107-d861-4467-bf8b-5e5ff18259f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:02.189841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01a107-d861-4467-bf8b-5e5ff18259f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:02.191613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01a107-d861-4467-bf8b-5e5ff18259f9 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:02.209676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1015/2007] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:28:02.211218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b98d8-0670-416c-82b1-01814de35343 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:02.223364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b98d8-0670-416c-82b1-01814de35343 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:02.229911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cb335-b935-45eb-8d55-a2b68bc9238a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:02.231351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cb335-b935-45eb-8d55-a2b68bc9238a admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.231765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cb335-b935-45eb-8d55-a2b68bc9238a admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.232140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cb335-b935-45eb-8d55-a2b68bc9238a admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.232912 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cb335-b935-45eb-8d55-a2b68bc9238a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:02.235116 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cb335-b935-45eb-8d55-a2b68bc9238a admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:02.337961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:02.344320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 993/2008] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:02 2022] POST /identity/v3/projects => generated 401 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:02.348343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01a107-d861-4467-bf8b-5e5ff18259f9 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:02.359696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01a107-d861-4467-bf8b-5e5ff18259f9 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:02.454741 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb3846-8013-47cc-87da-05c0b61d7cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:02.454990 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb3846-8013-47cc-87da-05c0b61d7cbb admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.455223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb3846-8013-47cc-87da-05c0b61d7cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.455502 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb3846-8013-47cc-87da-05c0b61d7cbb admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.456307 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb3846-8013-47cc-87da-05c0b61d7cbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:02.458355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb3846-8013-47cc-87da-05c0b61d7cbb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:02.557920 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1016/2009] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:02 2022] POST /identity/v3/users => generated 301 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:02.559119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cb335-b935-45eb-8d55-a2b68bc9238a admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:02.571731 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cb335-b935-45eb-8d55-a2b68bc9238a admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:02.574195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:02.579000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c10dbd-e393-4da2-855d-9116d37c0460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:02.583426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c10dbd-e393-4da2-855d-9116d37c0460 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.583426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c10dbd-e393-4da2-855d-9116d37c0460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.583426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c10dbd-e393-4da2-855d-9116d37c0460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.583426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c10dbd-e393-4da2-855d-9116d37c0460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:02.583426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c10dbd-e393-4da2-855d-9116d37c0460 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:02.593390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 994/2010] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:02.597562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb3846-8013-47cc-87da-05c0b61d7cbb admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:02.609023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb3846-8013-47cc-87da-05c0b61d7cbb admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:02.615520 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812d3d9-ea34-4e39-ba69-ff0603182430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:02.616492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812d3d9-ea34-4e39-ba69-ff0603182430 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.616899 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812d3d9-ea34-4e39-ba69-ff0603182430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.617226 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812d3d9-ea34-4e39-ba69-ff0603182430 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.617884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812d3d9-ea34-4e39-ba69-ff0603182430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:02.619523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812d3d9-ea34-4e39-ba69-ff0603182430 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:02.631141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1017/2011] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:28:02.635960 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c10dbd-e393-4da2-855d-9116d37c0460 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:02.637276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812d3d9-ea34-4e39-ba69-ff0603182430 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:02.638939 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-0812d3d9-ea34-4e39-ba69-ff0603182430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:02.652472 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c10dbd-e393-4da2-855d-9116d37c0460 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:02.654107 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812d3d9-ea34-4e39-ba69-ff0603182430 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:02.654214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5fd5fd-810d-4274-b90a-b3557dc5c3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:02.654709 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5fd5fd-810d-4274-b90a-b3557dc5c3ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.654986 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5fd5fd-810d-4274-b90a-b3557dc5c3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.655256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5fd5fd-810d-4274-b90a-b3557dc5c3ae admin admin] PATH_INFO: `/v3/projects/6432847b417645a087ff8e4ddc383898/users/c5019bb7e58b4661abc69ebe45f53b23/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.659217 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:02.659853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.660140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.660448 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] PATH_INFO: `/v3/projects/25f141f6504c4614ae7ea1e84c2cd187` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.662085 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=25f141f6504c4614ae7ea1e84c2cd187)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:02.664339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:02.682962 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5fd5fd-810d-4274-b90a-b3557dc5c3ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=6432847b417645a087ff8e4ddc383898, user_id=c5019bb7e58b4661abc69ebe45f53b23, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:02.687904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5fd5fd-810d-4274-b90a-b3557dc5c3ae admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:02.768289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 995/2012] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:02 2022] PUT /identity/v3/projects/6432847b417645a087ff8e4ddc383898/users/c5019bb7e58b4661abc69ebe45f53b23/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:02.778865 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5fd5fd-810d-4274-b90a-b3557dc5c3ae admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:02.778865 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ca5fd5fd-810d-4274-b90a-b3557dc5c3ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:02.794945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25f141f6504c4614ae7ea1e84c2cd187', 'request_id': 'req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:02.801667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25f141f6504c4614ae7ea1e84c2cd187', 'request_id': 'req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:02.810198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25f141f6504c4614ae7ea1e84c2cd187', 'request_id': 'req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:02.858537 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5fd5fd-810d-4274-b90a-b3557dc5c3ae admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:02.902531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] The token cache is being invalidate because project 25f141f6504c4614ae7ea1e84c2cd187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:02.902845 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:02.903557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:02.905120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1018/2013] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:02 2022] DELETE /identity/v3/projects/25f141f6504c4614ae7ea1e84c2cd187 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:02.905870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:02.907186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:02.920222 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:02.923461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.923461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.923461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c admin admin] PATH_INFO: `/v3/projects/c71576f2f02a4392848a14bde478e145` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.923461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c71576f2f02a4392848a14bde478e145)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:02.924646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:02.969870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bcd7a-3bcb-4da6-b61b-1f0c69889b4d admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:02.974989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:02.975614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:02.975896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:02.976189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] PATH_INFO: `/v3/projects/c43ecc2fc17946c28bf1ceb32cd6198d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:02.977330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c43ecc2fc17946c28bf1ceb32cd6198d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:02.979231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:03.069285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c71576f2f02a4392848a14bde478e145', 'request_id': 'req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.074609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c71576f2f02a4392848a14bde478e145', 'request_id': 'req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.154000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c43ecc2fc17946c28bf1ceb32cd6198d', 'request_id': 'req-db87dd13-8bc0-4ea5-b81a-523466940e24'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.160095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c43ecc2fc17946c28bf1ceb32cd6198d', 'request_id': 'req-db87dd13-8bc0-4ea5-b81a-523466940e24'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.165390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c43ecc2fc17946c28bf1ceb32cd6198d', 'request_id': 'req-db87dd13-8bc0-4ea5-b81a-523466940e24'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.232346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c71576f2f02a4392848a14bde478e145', 'request_id': 'req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.243680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c admin admin] The token cache is being invalidate because project c71576f2f02a4392848a14bde478e145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:03.243971 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.244650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3bdd7cd7-46d8-4cd7-88ba-c6903f25357c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.246155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 996/2014] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:02 2022] DELETE /identity/v3/projects/c71576f2f02a4392848a14bde478e145 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:03.247385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5f3bb-2867-4a7b-9052-6d8b1dcbab0c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:03.247673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5f3bb-2867-4a7b-9052-6d8b1dcbab0c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:03.247944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5f3bb-2867-4a7b-9052-6d8b1dcbab0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:03.275239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7ea5f3bb-2867-4a7b-9052-6d8b1dcbab0c None None] MFA Rules not processed for user `c5019bb7e58b4661abc69ebe45f53b23`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:03.276475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7ea5f3bb-2867-4a7b-9052-6d8b1dcbab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:03.336823 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] The token cache is being invalidate because project c43ecc2fc17946c28bf1ceb32cd6198d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:03.337007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.337715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.338418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 997/2015] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:03 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:03.339236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5f3bb-2867-4a7b-9052-6d8b1dcbab0c None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:03.341041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7ea5f3bb-2867-4a7b-9052-6d8b1dcbab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:03.343275 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1019/2016] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:02 2022] DELETE /identity/v3/projects/c43ecc2fc17946c28bf1ceb32cd6198d => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:03.354706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:03.354706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:03.405946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5f3bb-2867-4a7b-9052-6d8b1dcbab0c None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:03.409344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87dd13-8bc0-4ea5-b81a-523466940e24 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:03.410886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:03.411344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:03.411630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:03.411910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] PATH_INFO: `/v3/users/f3bdeae6ddc64eaf8751ed3b6e82a6b7` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:03.413064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3bdeae6ddc64eaf8751ed3b6e82a6b7)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:03.414138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249a391-9227-4459-9acc-3b8e4233ef92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:03.414900 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249a391-9227-4459-9acc-3b8e4233ef92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:03.414999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:03.415183 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249a391-9227-4459-9acc-3b8e4233ef92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:03.415484 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249a391-9227-4459-9acc-3b8e4233ef92 admin admin] PATH_INFO: `/v3/users/5988292c3335465e8c253edc0474680e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:03.416652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249a391-9227-4459-9acc-3b8e4233ef92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5988292c3335465e8c253edc0474680e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:03.419127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249a391-9227-4459-9acc-3b8e4233ef92 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:03.693579 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3bdeae6ddc64eaf8751ed3b6e82a6b7'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.694369 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3bdeae6ddc64eaf8751ed3b6e82a6b7'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.703754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3bdeae6ddc64eaf8751ed3b6e82a6b7'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.722260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f249a391-9227-4459-9acc-3b8e4233ef92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5988292c3335465e8c253edc0474680e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.728953 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f249a391-9227-4459-9acc-3b8e4233ef92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5988292c3335465e8c253edc0474680e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.846188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3bdeae6ddc64eaf8751ed3b6e82a6b7'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.846811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3bdeae6ddc64eaf8751ed3b6e82a6b7'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.864752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 998/2017] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:03 2022] DELETE /identity/v3/users/f3bdeae6ddc64eaf8751ed3b6e82a6b7 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:03.865513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:03.866933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:03.938920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7e86a-665e-4db5-9644-31841587dce8 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:03.960346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934402b-a6da-4202-a01a-e7cefc55da4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:03.960546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f249a391-9227-4459-9acc-3b8e4233ef92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5988292c3335465e8c253edc0474680e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.960636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934402b-a6da-4202-a01a-e7cefc55da4c admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:03.960636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934402b-a6da-4202-a01a-e7cefc55da4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:03.960636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934402b-a6da-4202-a01a-e7cefc55da4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:03.960636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934402b-a6da-4202-a01a-e7cefc55da4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:03.960921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f249a391-9227-4459-9acc-3b8e4233ef92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5988292c3335465e8c253edc0474680e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.960995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934402b-a6da-4202-a01a-e7cefc55da4c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:03.975709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f249a391-9227-4459-9acc-3b8e4233ef92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5988292c3335465e8c253edc0474680e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:03.982634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1020/2018] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:03 2022] DELETE /identity/v3/users/5988292c3335465e8c253edc0474680e => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:03.989399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6598e-956b-40bd-b325-b13023db1550 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:03.989399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6598e-956b-40bd-b325-b13023db1550 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:03.989399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6598e-956b-40bd-b325-b13023db1550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:04.005605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-9ee6598e-956b-40bd-b325-b13023db1550 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:04.007599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9ee6598e-956b-40bd-b325-b13023db1550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:04.075422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1021/2019] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:04.085785 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200914ab-18ea-4d85-8dc9-5feb6af828d5 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:04.086076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200914ab-18ea-4d85-8dc9-5feb6af828d5 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:04.086358 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200914ab-18ea-4d85-8dc9-5feb6af828d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:04.108817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-200914ab-18ea-4d85-8dc9-5feb6af828d5 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:04.110892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:04.111776 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-200914ab-18ea-4d85-8dc9-5feb6af828d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:04.119832 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1022/2020] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:04.124421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200914ab-18ea-4d85-8dc9-5feb6af828d5 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:04.133030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200914ab-18ea-4d85-8dc9-5feb6af828d5 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:04.138109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b16d79-bd5a-4c73-a558-42995aa81d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:04.138689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b16d79-bd5a-4c73-a558-42995aa81d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:04.138965 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b16d79-bd5a-4c73-a558-42995aa81d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:04.139236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b16d79-bd5a-4c73-a558-42995aa81d24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:04.139883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b16d79-bd5a-4c73-a558-42995aa81d24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:04.140211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b16d79-bd5a-4c73-a558-42995aa81d24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:04.141813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b16d79-bd5a-4c73-a558-42995aa81d24 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:04.153144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1023/2021] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:28:04.157003 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b16d79-bd5a-4c73-a558-42995aa81d24 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:04.166357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b16d79-bd5a-4c73-a558-42995aa81d24 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:04.169244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 999/2022] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:03 2022] POST /identity/v3/projects => generated 399 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:04.175126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934402b-a6da-4202-a01a-e7cefc55da4c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:04.176951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4934402b-a6da-4202-a01a-e7cefc55da4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:04.233274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba1d08-e970-4fc4-ade0-7bc308ca7510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:04.233927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba1d08-e970-4fc4-ade0-7bc308ca7510 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:04.234204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba1d08-e970-4fc4-ade0-7bc308ca7510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:04.234476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba1d08-e970-4fc4-ade0-7bc308ca7510 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:04.235116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba1d08-e970-4fc4-ade0-7bc308ca7510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:04.237649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba1d08-e970-4fc4-ade0-7bc308ca7510 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:04.261105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934402b-a6da-4202-a01a-e7cefc55da4c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:04.267055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf27759-affa-41b1-959e-c9a1dd050dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:04.268288 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf27759-affa-41b1-959e-c9a1dd050dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:04.268652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf27759-affa-41b1-959e-c9a1dd050dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:04.269474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf27759-affa-41b1-959e-c9a1dd050dd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:04.269707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf27759-affa-41b1-959e-c9a1dd050dd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:04.271720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf27759-affa-41b1-959e-c9a1dd050dd5 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:04.426980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1000/2023] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:04 2022] POST /identity/v3/users => generated 306 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:04.427186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1024/2024] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:04 2022] POST /identity/v3/projects => generated 375 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:04.428536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf27759-affa-41b1-959e-c9a1dd050dd5 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:04.429555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba1d08-e970-4fc4-ade0-7bc308ca7510 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:04.439891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf27759-affa-41b1-959e-c9a1dd050dd5 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:04.494376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba1d08-e970-4fc4-ade0-7bc308ca7510 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:04.499813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9b68d-2124-4981-9bb7-44f454a2ba25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:04.500464 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9b68d-2124-4981-9bb7-44f454a2ba25 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:04.501435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9b68d-2124-4981-9bb7-44f454a2ba25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:04.501707 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9b68d-2124-4981-9bb7-44f454a2ba25 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:04.502364 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9b68d-2124-4981-9bb7-44f454a2ba25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:04.504316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9b68d-2124-4981-9bb7-44f454a2ba25 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:04.509195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba9979-4f0a-495a-bff9-3dbe72b2cf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:04.509823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba9979-4f0a-495a-bff9-3dbe72b2cf06 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:04.510107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba9979-4f0a-495a-bff9-3dbe72b2cf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:04.510370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba9979-4f0a-495a-bff9-3dbe72b2cf06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:04.511021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba9979-4f0a-495a-bff9-3dbe72b2cf06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:04.513394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba9979-4f0a-495a-bff9-3dbe72b2cf06 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:04.525891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1001/2025] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:04.526793 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba9979-4f0a-495a-bff9-3dbe72b2cf06 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:04.529875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c3ba9979-4f0a-495a-bff9-3dbe72b2cf06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:04.546808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba9979-4f0a-495a-bff9-3dbe72b2cf06 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:04.552116 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:04.553091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:04.553354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:04.553618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] PATH_INFO: `/v3/projects/0357f16c07694a8d984c6f592d04a61e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:04.554938 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0357f16c07694a8d984c6f592d04a61e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:04.556980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:04.635037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1025/2026] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:04 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:04.636001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9b68d-2124-4981-9bb7-44f454a2ba25 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:04.645913 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9b68d-2124-4981-9bb7-44f454a2ba25 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:04.650575 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a0a56-8752-4622-8115-689bff101a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:04.651202 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a0a56-8752-4622-8115-689bff101a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:04.651506 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a0a56-8752-4622-8115-689bff101a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:04.651779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a0a56-8752-4622-8115-689bff101a72 admin admin] PATH_INFO: `/v3/projects/934517c69faf40d9a402ec644849e1d9/users/fd5d45e519f8454f9823e35c3e0e871f/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:04.671598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a0a56-8752-4622-8115-689bff101a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=934517c69faf40d9a402ec644849e1d9, user_id=fd5d45e519f8454f9823e35c3e0e871f, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:04.671598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a0a56-8752-4622-8115-689bff101a72 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:04.812634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1026/2027] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:04 2022] PUT /identity/v3/projects/934517c69faf40d9a402ec644849e1d9/users/fd5d45e519f8454f9823e35c3e0e871f/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:04.815331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a0a56-8752-4622-8115-689bff101a72 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:04.817177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-862a0a56-8752-4622-8115-689bff101a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:04.823486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0357f16c07694a8d984c6f592d04a61e', 'request_id': 'req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:04.836386 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0357f16c07694a8d984c6f592d04a61e', 'request_id': 'req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:04.893853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a0a56-8752-4622-8115-689bff101a72 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:04.952435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:04.953080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:04.953386 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:04.953660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] PATH_INFO: `/v3/projects/df732d28d4bb492a972eb7623268fd76` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:04.954836 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df732d28d4bb492a972eb7623268fd76)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:04.956939 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:04.979246 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0357f16c07694a8d984c6f592d04a61e', 'request_id': 'req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:04.991965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] The token cache is being invalidate because project 0357f16c07694a8d984c6f592d04a61e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:04.992302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:04.992969 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:04.994502 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1002/2028] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:04 2022] DELETE /identity/v3/projects/0357f16c07694a8d984c6f592d04a61e => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:04.995450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:04.997329 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:05.010111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec2df2-42a3-4ae2-a7c3-432a575c9d0b admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.015226 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9ec37-3d5b-4cca-a4b8-5fb61641a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.015890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9ec37-3d5b-4cca-a4b8-5fb61641a192 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.016127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9ec37-3d5b-4cca-a4b8-5fb61641a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.016437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9ec37-3d5b-4cca-a4b8-5fb61641a192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.017088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9ec37-3d5b-4cca-a4b8-5fb61641a192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:05.018923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9ec37-3d5b-4cca-a4b8-5fb61641a192 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:05.028763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1003/2029] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:05.029679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9ec37-3d5b-4cca-a4b8-5fb61641a192 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:05.031252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e7c9ec37-3d5b-4cca-a4b8-5fb61641a192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:05.043399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9ec37-3d5b-4cca-a4b8-5fb61641a192 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.048023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c2834-55c9-421b-b022-9484775b6387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.048618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c2834-55c9-421b-b022-9484775b6387 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.048900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c2834-55c9-421b-b022-9484775b6387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.049176 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c2834-55c9-421b-b022-9484775b6387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.049789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c2834-55c9-421b-b022-9484775b6387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:05.051392 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c2834-55c9-421b-b022-9484775b6387 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:05.061688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1004/2030] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:05.062884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292f1b-45a7-4520-ba0c-430acbf9e4d3 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.063173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292f1b-45a7-4520-ba0c-430acbf9e4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.063444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292f1b-45a7-4520-ba0c-430acbf9e4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.082536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-65292f1b-45a7-4520-ba0c-430acbf9e4d3 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:05.083743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-65292f1b-45a7-4520-ba0c-430acbf9e4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:05.091765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1005/2031] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:05.092568 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65292f1b-45a7-4520-ba0c-430acbf9e4d3 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:05.102057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65292f1b-45a7-4520-ba0c-430acbf9e4d3 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.103410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df732d28d4bb492a972eb7623268fd76', 'request_id': 'req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:05.110267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df732d28d4bb492a972eb7623268fd76', 'request_id': 'req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:05.128802 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df732d28d4bb492a972eb7623268fd76', 'request_id': 'req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:05.161576 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea607d4e-c988-4396-be55-a08e2db2edf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.162185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607d4e-c988-4396-be55-a08e2db2edf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.162468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607d4e-c988-4396-be55-a08e2db2edf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.162742 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607d4e-c988-4396-be55-a08e2db2edf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.163389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea607d4e-c988-4396-be55-a08e2db2edf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:05.165016 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea607d4e-c988-4396-be55-a08e2db2edf2 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:05.175359 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1006/2032] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:05.176138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea607d4e-c988-4396-be55-a08e2db2edf2 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:05.186311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea607d4e-c988-4396-be55-a08e2db2edf2 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.191422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b9df6-018e-46d3-b1bf-3ec7b7e34ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.192062 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b9df6-018e-46d3-b1bf-3ec7b7e34ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.192371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b9df6-018e-46d3-b1bf-3ec7b7e34ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.192652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b9df6-018e-46d3-b1bf-3ec7b7e34ce1 admin admin] PATH_INFO: `/v3/projects/934517c69faf40d9a402ec644849e1d9/users/fd5d45e519f8454f9823e35c3e0e871f/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.194527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b9df6-018e-46d3-b1bf-3ec7b7e34ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=934517c69faf40d9a402ec644849e1d9, user_id=fd5d45e519f8454f9823e35c3e0e871f, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:05.197189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b9df6-018e-46d3-b1bf-3ec7b7e34ce1 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:05.208070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1007/2033] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:05 2022] PUT /identity/v3/projects/934517c69faf40d9a402ec644849e1d9/users/fd5d45e519f8454f9823e35c3e0e871f/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:05.209370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54947fcd-06f6-4927-93e4-2932af3d39a8 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.209883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54947fcd-06f6-4927-93e4-2932af3d39a8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.209982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54947fcd-06f6-4927-93e4-2932af3d39a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.230490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-54947fcd-06f6-4927-93e4-2932af3d39a8 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:05.232544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-54947fcd-06f6-4927-93e4-2932af3d39a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:05.245772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] The token cache is being invalidate because project df732d28d4bb492a972eb7623268fd76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:05.245772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:05.245772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:05.245772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1027/2034] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:04 2022] DELETE /identity/v3/projects/df732d28d4bb492a972eb7623268fd76 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:05.245772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:05.248694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:05.262695 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca009a9-1f87-4ae5-9d0b-1ee4ba079793 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.322809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1008/2035] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:05.326798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8f7d4-a45d-4b6c-9608-3f65d7969d64 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.326798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8f7d4-a45d-4b6c-9608-3f65d7969d64 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.326798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8f7d4-a45d-4b6c-9608-3f65d7969d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.338605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9853e-42f6-42d6-80d1-6c51f80be787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.338605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9853e-42f6-42d6-80d1-6c51f80be787 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.338812 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9853e-42f6-42d6-80d1-6c51f80be787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.339066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9853e-42f6-42d6-80d1-6c51f80be787 admin admin] PATH_INFO: `/v3/projects/1cd24239272b44388ddb1304fb184380/users/b09be3e57c644c938aae47464c9832d7/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.354794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9853e-42f6-42d6-80d1-6c51f80be787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd24239272b44388ddb1304fb184380, user_id=b09be3e57c644c938aae47464c9832d7, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:05.356853 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f9f8f7d4-a45d-4b6c-9608-3f65d7969d64 None None] MFA Rules not processed for user `fd5d45e519f8454f9823e35c3e0e871f`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:05.358055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f9f8f7d4-a45d-4b6c-9608-3f65d7969d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:05.358295 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9853e-42f6-42d6-80d1-6c51f80be787 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:05.362036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:05.433354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1009/2036] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:05 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:05.434770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb546d6-4401-4010-a1f8-95dedc960467 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.435069 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb546d6-4401-4010-a1f8-95dedc960467 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.435346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb546d6-4401-4010-a1f8-95dedc960467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.455100 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-fdb546d6-4401-4010-a1f8-95dedc960467 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:05.456384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-fdb546d6-4401-4010-a1f8-95dedc960467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:05.464201 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1010/2037] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:05.464959 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb546d6-4401-4010-a1f8-95dedc960467 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:05.474498 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb546d6-4401-4010-a1f8-95dedc960467 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.478951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8001d-e3a2-4e39-898a-edd475863b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.481719 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8001d-e3a2-4e39-898a-edd475863b1c admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.481719 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8001d-e3a2-4e39-898a-edd475863b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.481719 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8001d-e3a2-4e39-898a-edd475863b1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.481719 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8001d-e3a2-4e39-898a-edd475863b1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:05.481719 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8001d-e3a2-4e39-898a-edd475863b1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:05.483075 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8001d-e3a2-4e39-898a-edd475863b1c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:05.494365 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1011/2038] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:28:05.495123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8001d-e3a2-4e39-898a-edd475863b1c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:05.510933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-73d8001d-e3a2-4e39-898a-edd475863b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:05.511091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1028/2039] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:05 2022] PUT /identity/v3/projects/1cd24239272b44388ddb1304fb184380/users/b09be3e57c644c938aae47464c9832d7/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:05.511091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745f419-9ff1-4fa5-816e-b237726fecf2 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.511091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745f419-9ff1-4fa5-816e-b237726fecf2 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.511091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745f419-9ff1-4fa5-816e-b237726fecf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.524942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-8745f419-9ff1-4fa5-816e-b237726fecf2 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:05.526157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8745f419-9ff1-4fa5-816e-b237726fecf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:05.558679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8001d-e3a2-4e39-898a-edd475863b1c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.587693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1029/2040] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:05.588312 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745f419-9ff1-4fa5-816e-b237726fecf2 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:05.597766 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745f419-9ff1-4fa5-816e-b237726fecf2 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.602396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11ff84-f42f-474a-ba5f-ecb32b418f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.603001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11ff84-f42f-474a-ba5f-ecb32b418f17 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.603295 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11ff84-f42f-474a-ba5f-ecb32b418f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.603564 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11ff84-f42f-474a-ba5f-ecb32b418f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.604227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11ff84-f42f-474a-ba5f-ecb32b418f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:05.606204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11ff84-f42f-474a-ba5f-ecb32b418f17 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:05.616495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f36db-cdb1-495b-a53f-f34d52f615b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.617117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f36db-cdb1-495b-a53f-f34d52f615b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.617796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f36db-cdb1-495b-a53f-f34d52f615b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.617796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f36db-cdb1-495b-a53f-f34d52f615b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.679390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f36db-cdb1-495b-a53f-f34d52f615b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:05.680862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f36db-cdb1-495b-a53f-f34d52f615b3 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:05.691844 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1012/2041] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:05 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:05.693101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad06f-12d7-46d2-b9c7-8caf3dfe2fbf None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.693384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad06f-12d7-46d2-b9c7-8caf3dfe2fbf None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.693654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad06f-12d7-46d2-b9c7-8caf3dfe2fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.724714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1030/2042] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:05 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:05.724910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-bd5ad06f-12d7-46d2-b9c7-8caf3dfe2fbf None None] MFA Rules not processed for user `b09be3e57c644c938aae47464c9832d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:05.725001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11ff84-f42f-474a-ba5f-ecb32b418f17 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:05.725083 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-bd5ad06f-12d7-46d2-b9c7-8caf3dfe2fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:05.785767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11ff84-f42f-474a-ba5f-ecb32b418f17 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.790513 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc50fa3-c431-4f4e-b9c5-ccef4be97a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.790688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1013/2043] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:05.791320 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc50fa3-c431-4f4e-b9c5-ccef4be97a31 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.791445 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ad06f-12d7-46d2-b9c7-8caf3dfe2fbf None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:05.791767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc50fa3-c431-4f4e-b9c5-ccef4be97a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.792134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc50fa3-c431-4f4e-b9c5-ccef4be97a31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.792910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc50fa3-c431-4f4e-b9c5-ccef4be97a31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:05.793343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc50fa3-c431-4f4e-b9c5-ccef4be97a31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:05.795139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc50fa3-c431-4f4e-b9c5-ccef4be97a31 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:05.804570 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ad06f-12d7-46d2-b9c7-8caf3dfe2fbf None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.806784 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1031/2044] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:28:05.807623 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc50fa3-c431-4f4e-b9c5-ccef4be97a31 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:05.810135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cbc50fa3-c431-4f4e-b9c5-ccef4be97a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:05.810994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b16dbf-cc13-4875-a61c-daff080631d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.812456 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16dbf-cc13-4875-a61c-daff080631d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.812768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16dbf-cc13-4875-a61c-daff080631d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.813110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16dbf-cc13-4875-a61c-daff080631d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.813829 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b16dbf-cc13-4875-a61c-daff080631d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:05.815883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b16dbf-cc13-4875-a61c-daff080631d9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:05.867619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc50fa3-c431-4f4e-b9c5-ccef4be97a31 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.871911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12305d06-1f04-4ab5-a67b-ac1bb5c01b5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.872555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12305d06-1f04-4ab5-a67b-ac1bb5c01b5b service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.872838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12305d06-1f04-4ab5-a67b-ac1bb5c01b5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.873110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12305d06-1f04-4ab5-a67b-ac1bb5c01b5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.881479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12305d06-1f04-4ab5-a67b-ac1bb5c01b5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:05.882970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12305d06-1f04-4ab5-a67b-ac1bb5c01b5b service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:05.892285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1032/2045] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:05.893392 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12305d06-1f04-4ab5-a67b-ac1bb5c01b5b service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:05.903690 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12305d06-1f04-4ab5-a67b-ac1bb5c01b5b service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.908406 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c171e-d181-4cfd-8b63-5888b48b0581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.908977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c171e-d181-4cfd-8b63-5888b48b0581 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.909259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c171e-d181-4cfd-8b63-5888b48b0581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.909533 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c171e-d181-4cfd-8b63-5888b48b0581 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.910144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c171e-d181-4cfd-8b63-5888b48b0581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:05.912022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c171e-d181-4cfd-8b63-5888b48b0581 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:05.924935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1014/2046] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:05 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:05.926289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b16dbf-cc13-4875-a61c-daff080631d9 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:05.937103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b16dbf-cc13-4875-a61c-daff080631d9 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.942766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072059f-f87d-4180-a4c1-666f0081dfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.945646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072059f-f87d-4180-a4c1-666f0081dfc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.945646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072059f-f87d-4180-a4c1-666f0081dfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.945646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072059f-f87d-4180-a4c1-666f0081dfc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.945646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072059f-f87d-4180-a4c1-666f0081dfc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:05.946770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072059f-f87d-4180-a4c1-666f0081dfc0 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:05.968793 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1015/2047] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:05.970295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072059f-f87d-4180-a4c1-666f0081dfc0 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:05.981220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072059f-f87d-4180-a4c1-666f0081dfc0 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:05.985291 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0812a-1da8-4a51-887a-3450c7070adf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:05.985973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0812a-1da8-4a51-887a-3450c7070adf service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:05.986269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0812a-1da8-4a51-887a-3450c7070adf service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:05.986546 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0812a-1da8-4a51-887a-3450c7070adf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:05.988513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3de0812a-1da8-4a51-887a-3450c7070adf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:06.031763 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1033/2048] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:05 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:06.032661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c171e-d181-4cfd-8b63-5888b48b0581 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:06.058613 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0812a-1da8-4a51-887a-3450c7070adf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:06.060668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0812a-1da8-4a51-887a-3450c7070adf service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:06.076380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1016/2049] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:05 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:06.077454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0812a-1da8-4a51-887a-3450c7070adf service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:06.091493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0812a-1da8-4a51-887a-3450c7070adf service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:06.107404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c171e-d181-4cfd-8b63-5888b48b0581 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:06.112806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff678c91-487e-4cc9-91c9-e7eeb0e70529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:06.113409 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff678c91-487e-4cc9-91c9-e7eeb0e70529 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:06.113746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff678c91-487e-4cc9-91c9-e7eeb0e70529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:06.114031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff678c91-487e-4cc9-91c9-e7eeb0e70529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:06.114683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff678c91-487e-4cc9-91c9-e7eeb0e70529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:06.116445 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff678c91-487e-4cc9-91c9-e7eeb0e70529 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:06.132778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1034/2050] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:06.133723 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff678c91-487e-4cc9-91c9-e7eeb0e70529 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:06.144601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff678c91-487e-4cc9-91c9-e7eeb0e70529 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:06.149862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de57b14-92e9-4fd4-a07e-e728cf61e033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:06.150533 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de57b14-92e9-4fd4-a07e-e728cf61e033 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:06.150819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de57b14-92e9-4fd4-a07e-e728cf61e033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:06.151099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de57b14-92e9-4fd4-a07e-e728cf61e033 admin admin] PATH_INFO: `/v3/projects/e19c5148270a4b7697391ea799683636/users/f8dbf240ad1743c7ac152ce8ddd29213/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:06.167907 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de57b14-92e9-4fd4-a07e-e728cf61e033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19c5148270a4b7697391ea799683636, user_id=f8dbf240ad1743c7ac152ce8ddd29213, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:06.171309 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de57b14-92e9-4fd4-a07e-e728cf61e033 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:06.174154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31489f-ba26-43cf-905c-c261fa6b05b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:06.174776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31489f-ba26-43cf-905c-c261fa6b05b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:06.175056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31489f-ba26-43cf-905c-c261fa6b05b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:06.175978 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31489f-ba26-43cf-905c-c261fa6b05b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:06.175978 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31489f-ba26-43cf-905c-c261fa6b05b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:06.178163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31489f-ba26-43cf-905c-c261fa6b05b9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:06.240269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1035/2051] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:06 2022] PUT /identity/v3/projects/e19c5148270a4b7697391ea799683636/users/f8dbf240ad1743c7ac152ce8ddd29213/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:06.250294 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7ec93-7335-4d6a-b2bd-b5286d919758 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:06.250581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7ec93-7335-4d6a-b2bd-b5286d919758 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:06.250859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7ec93-7335-4d6a-b2bd-b5286d919758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:06.278981 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-9dd7ec93-7335-4d6a-b2bd-b5286d919758 None None] MFA Rules not processed for user `f8dbf240ad1743c7ac152ce8ddd29213`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:06.280205 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9dd7ec93-7335-4d6a-b2bd-b5286d919758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:06.315753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1017/2052] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:06 2022] POST /identity/v3/users => generated 287 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:06.319506 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31489f-ba26-43cf-905c-c261fa6b05b9 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:06.331484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31489f-ba26-43cf-905c-c261fa6b05b9 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:06.363736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1036/2053] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:28:06.368027 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7ec93-7335-4d6a-b2bd-b5286d919758 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:06.399873 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2cf12-f8ea-4780-af43-54e5da591da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:06.400677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2cf12-f8ea-4780-af43-54e5da591da1 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:06.401080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2cf12-f8ea-4780-af43-54e5da591da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:06.401460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2cf12-f8ea-4780-af43-54e5da591da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:06.402182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2cf12-f8ea-4780-af43-54e5da591da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:06.404206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2cf12-f8ea-4780-af43-54e5da591da1 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:06.418145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1018/2054] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:28:06.419493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2cf12-f8ea-4780-af43-54e5da591da1 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:06.430279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2cf12-f8ea-4780-af43-54e5da591da1 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:06.435847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb431e1-c618-4429-b007-151c81b80653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:06.440683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7ec93-7335-4d6a-b2bd-b5286d919758 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:06.440818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb431e1-c618-4429-b007-151c81b80653 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:06.440818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb431e1-c618-4429-b007-151c81b80653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:06.440818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb431e1-c618-4429-b007-151c81b80653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:06.440818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb431e1-c618-4429-b007-151c81b80653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:06.440818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb431e1-c618-4429-b007-151c81b80653 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:06.441155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f867100-7fee-4508-a0a4-d7e1015d9eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:06.441252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f867100-7fee-4508-a0a4-d7e1015d9eed admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:06.441853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f867100-7fee-4508-a0a4-d7e1015d9eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:06.441853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f867100-7fee-4508-a0a4-d7e1015d9eed admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:06.442420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f867100-7fee-4508-a0a4-d7e1015d9eed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:06.444394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f867100-7fee-4508-a0a4-d7e1015d9eed admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:06.453514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1019/2055] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:06.454530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb431e1-c618-4429-b007-151c81b80653 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:06.469677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb431e1-c618-4429-b007-151c81b80653 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:06.481309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc122e-7c48-4685-92f2-e50883fe70b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:06.482178 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc122e-7c48-4685-92f2-e50883fe70b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:06.482602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc122e-7c48-4685-92f2-e50883fe70b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:06.483673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc122e-7c48-4685-92f2-e50883fe70b0 admin admin] PATH_INFO: `/v3/projects/6557d6a51ff94049be43d40321c09526/users/c2abe65262604d89adc92ff86b1748c1/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:06.500536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc122e-7c48-4685-92f2-e50883fe70b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6557d6a51ff94049be43d40321c09526, user_id=c2abe65262604d89adc92ff86b1748c1, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:06.503619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc122e-7c48-4685-92f2-e50883fe70b0 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:06.669520 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:06.705703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1037/2056] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:06 2022] POST /identity/v3/projects => generated 377 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:06.706517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1020/2057] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:06 2022] PUT /identity/v3/projects/6557d6a51ff94049be43d40321c09526/users/c2abe65262604d89adc92ff86b1748c1/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:06.710009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f867100-7fee-4508-a0a4-d7e1015d9eed admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:06.713200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f624403-4c97-40fa-a704-ac64b421d6b8 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:06.713483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f624403-4c97-40fa-a704-ac64b421d6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:06.713812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f624403-4c97-40fa-a704-ac64b421d6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:06.737714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-0f624403-4c97-40fa-a704-ac64b421d6b8 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:06.738930 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0f624403-4c97-40fa-a704-ac64b421d6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:06.778810 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f867100-7fee-4508-a0a4-d7e1015d9eed admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:06.783421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101e734-969e-4445-9cb9-21c2c6b95aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:06.784038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101e734-969e-4445-9cb9-21c2c6b95aff admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:06.784358 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101e734-969e-4445-9cb9-21c2c6b95aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:06.784637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101e734-969e-4445-9cb9-21c2c6b95aff admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:06.785285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101e734-969e-4445-9cb9-21c2c6b95aff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:06.787254 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101e734-969e-4445-9cb9-21c2c6b95aff admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:06.798397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1021/2058] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:06.808194 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f624403-4c97-40fa-a704-ac64b421d6b8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:06.810182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0f624403-4c97-40fa-a704-ac64b421d6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:06.866926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f624403-4c97-40fa-a704-ac64b421d6b8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:06.870636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746bcee-65d7-454f-a1d0-87601040c6cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:06.871906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746bcee-65d7-454f-a1d0-87601040c6cd service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:06.872239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746bcee-65d7-454f-a1d0-87601040c6cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:06.872517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746bcee-65d7-454f-a1d0-87601040c6cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:06.881787 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746bcee-65d7-454f-a1d0-87601040c6cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:06.883242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746bcee-65d7-454f-a1d0-87601040c6cd service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:06.895505 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1022/2059] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:28:06.949335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1038/2060] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:06 2022] POST /identity/v3/users => generated 295 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:06.953483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746bcee-65d7-454f-a1d0-87601040c6cd service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:06.963030 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746bcee-65d7-454f-a1d0-87601040c6cd service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:06.967709 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ed283-9d54-4cf7-b176-ab6b30c0229c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:06.968382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ed283-9d54-4cf7-b176-ab6b30c0229c admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:06.968652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ed283-9d54-4cf7-b176-ab6b30c0229c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:06.968929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ed283-9d54-4cf7-b176-ab6b30c0229c admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:06.969568 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ed283-9d54-4cf7-b176-ab6b30c0229c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:06.971161 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ed283-9d54-4cf7-b176-ab6b30c0229c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:06.982310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1023/2061] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:06.986535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101e734-969e-4445-9cb9-21c2c6b95aff admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:06.997714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101e734-969e-4445-9cb9-21c2c6b95aff admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:07.002638 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb0c48-d9e5-4120-8408-135fe48dcfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:07.003296 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb0c48-d9e5-4120-8408-135fe48dcfb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:07.003576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb0c48-d9e5-4120-8408-135fe48dcfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:07.003850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb0c48-d9e5-4120-8408-135fe48dcfb7 admin admin] PATH_INFO: `/v3/projects/587064eed23247109ba317e4dda2ef25/users/3d8b4954e24a498fb12bf938f74b29c5/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:07.019228 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb0c48-d9e5-4120-8408-135fe48dcfb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587064eed23247109ba317e4dda2ef25, user_id=3d8b4954e24a498fb12bf938f74b29c5, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:07.021997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb0c48-d9e5-4120-8408-135fe48dcfb7 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:07.110599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1039/2062] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:06 2022] PUT /identity/v3/projects/587064eed23247109ba317e4dda2ef25/users/3d8b4954e24a498fb12bf938f74b29c5/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:07.115207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ed283-9d54-4cf7-b176-ab6b30c0229c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:07.132606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ed283-9d54-4cf7-b176-ab6b30c0229c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:07.275590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf7b01-26c2-426e-abc4-2d52af094dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:07.276390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf7b01-26c2-426e-abc4-2d52af094dad admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:07.276775 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf7b01-26c2-426e-abc4-2d52af094dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:07.277150 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf7b01-26c2-426e-abc4-2d52af094dad admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:07.277923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf7b01-26c2-426e-abc4-2d52af094dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:07.279602 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf7b01-26c2-426e-abc4-2d52af094dad admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:07.292250 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1024/2063] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:07.297930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb0c48-d9e5-4120-8408-135fe48dcfb7 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:07.309774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb0c48-d9e5-4120-8408-135fe48dcfb7 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:07.315094 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca7b36-ae92-4057-b812-e81676dbeaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:07.315767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca7b36-ae92-4057-b812-e81676dbeaf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:07.316038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca7b36-ae92-4057-b812-e81676dbeaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:07.316384 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca7b36-ae92-4057-b812-e81676dbeaf6 admin admin] PATH_INFO: `/v3/projects/587064eed23247109ba317e4dda2ef25/users/3d8b4954e24a498fb12bf938f74b29c5/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:07.318267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca7b36-ae92-4057-b812-e81676dbeaf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587064eed23247109ba317e4dda2ef25, user_id=3d8b4954e24a498fb12bf938f74b29c5, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:07.321009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca7b36-ae92-4057-b812-e81676dbeaf6 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:07.332721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1040/2064] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:07 2022] PUT /identity/v3/projects/587064eed23247109ba317e4dda2ef25/users/3d8b4954e24a498fb12bf938f74b29c5/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:07.349937 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4fa58-14d1-48fc-b345-199d3181f61d None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:07.350634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4fa58-14d1-48fc-b345-199d3181f61d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:07.350926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4fa58-14d1-48fc-b345-199d3181f61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:07.382515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-20c4fa58-14d1-48fc-b345-199d3181f61d None None] MFA Rules not processed for user `3d8b4954e24a498fb12bf938f74b29c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:07.383710 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-20c4fa58-14d1-48fc-b345-199d3181f61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:07.510290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1025/2065] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:07.521916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca7b36-ae92-4057-b812-e81676dbeaf6 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:07.586319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca7b36-ae92-4057-b812-e81676dbeaf6 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:07.590104 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272537bb-3ac3-4d8d-b96a-dab513e89b76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:07.591594 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272537bb-3ac3-4d8d-b96a-dab513e89b76 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:07.591885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272537bb-3ac3-4d8d-b96a-dab513e89b76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:07.592185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272537bb-3ac3-4d8d-b96a-dab513e89b76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:07.665717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272537bb-3ac3-4d8d-b96a-dab513e89b76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:07.667374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272537bb-3ac3-4d8d-b96a-dab513e89b76 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:07.678019 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1041/2066] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:07.693718 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:08.018770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:08.505776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72cd91-f19b-41e6-89b2-c471cc8265e8 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:08.506295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72cd91-f19b-41e6-89b2-c471cc8265e8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:08.506402 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72cd91-f19b-41e6-89b2-c471cc8265e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:08.538215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-9b72cd91-f19b-41e6-89b2-c471cc8265e8 None None] MFA Rules not processed for user `c2abe65262604d89adc92ff86b1748c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:08.539452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9b72cd91-f19b-41e6-89b2-c471cc8265e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:08.617795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1026/2067] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:08.622180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272537bb-3ac3-4d8d-b96a-dab513e89b76 service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:08.695165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272537bb-3ac3-4d8d-b96a-dab513e89b76 service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:08.699767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15efa4-c106-47a2-9253-19cbdeea32f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:08.700399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15efa4-c106-47a2-9253-19cbdeea32f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:08.700679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15efa4-c106-47a2-9253-19cbdeea32f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:08.700949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15efa4-c106-47a2-9253-19cbdeea32f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:08.701577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15efa4-c106-47a2-9253-19cbdeea32f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:08.703504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15efa4-c106-47a2-9253-19cbdeea32f1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:08.896213 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1042/2068] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:08 2022] POST /identity/v3/projects => generated 373 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:08.899080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72cd91-f19b-41e6-89b2-c471cc8265e8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:08.910311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72cd91-f19b-41e6-89b2-c471cc8265e8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:08.985398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae33d5c-3642-47f4-8611-c54f6d6832aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:08.986181 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae33d5c-3642-47f4-8611-c54f6d6832aa admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:08.986462 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae33d5c-3642-47f4-8611-c54f6d6832aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:08.989284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae33d5c-3642-47f4-8611-c54f6d6832aa admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:08.989284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae33d5c-3642-47f4-8611-c54f6d6832aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:08.989474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae33d5c-3642-47f4-8611-c54f6d6832aa admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:09.252637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1027/2069] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:08 2022] POST /identity/v3/users => generated 293 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:09.256551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15efa4-c106-47a2-9253-19cbdeea32f1 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:09.266591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15efa4-c106-47a2-9253-19cbdeea32f1 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:09.271415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e93a2e-95f9-4599-ad0c-fe7be5de1fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:09.272022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e93a2e-95f9-4599-ad0c-fe7be5de1fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:09.272331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e93a2e-95f9-4599-ad0c-fe7be5de1fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:09.272621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e93a2e-95f9-4599-ad0c-fe7be5de1fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:09.273264 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e93a2e-95f9-4599-ad0c-fe7be5de1fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:09.274865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e93a2e-95f9-4599-ad0c-fe7be5de1fb2 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:09.285713 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1043/2070] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:28:09.296893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae33d5c-3642-47f4-8611-c54f6d6832aa admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:09.306772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae33d5c-3642-47f4-8611-c54f6d6832aa admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:09.311529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237863a0-c037-49c7-926b-aac3737c66ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:09.312197 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237863a0-c037-49c7-926b-aac3737c66ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:09.312562 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237863a0-c037-49c7-926b-aac3737c66ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:09.312836 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237863a0-c037-49c7-926b-aac3737c66ad admin admin] PATH_INFO: `/v3/projects/9cba124a13c44703a0625f8d18cb5037/users/3c8766948d6845cb89a7be69a8a88601/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:09.327154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237863a0-c037-49c7-926b-aac3737c66ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cba124a13c44703a0625f8d18cb5037, user_id=3c8766948d6845cb89a7be69a8a88601, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:09.329885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237863a0-c037-49c7-926b-aac3737c66ad admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:09.480595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1028/2071] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:09 2022] PUT /identity/v3/projects/9cba124a13c44703a0625f8d18cb5037/users/3c8766948d6845cb89a7be69a8a88601/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:09.491661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e93a2e-95f9-4599-ad0c-fe7be5de1fb2 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:09.501744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e93a2e-95f9-4599-ad0c-fe7be5de1fb2 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:09.538322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:09.560951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e06d50-80fe-4b47-90e7-18b3f8b070bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:09.560951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e06d50-80fe-4b47-90e7-18b3f8b070bf admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:09.561468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e06d50-80fe-4b47-90e7-18b3f8b070bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:09.561754 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e06d50-80fe-4b47-90e7-18b3f8b070bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:09.562396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e06d50-80fe-4b47-90e7-18b3f8b070bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:09.563989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e06d50-80fe-4b47-90e7-18b3f8b070bf admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:09.573884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1044/2072] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:09.578157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237863a0-c037-49c7-926b-aac3737c66ad admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:09.588962 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237863a0-c037-49c7-926b-aac3737c66ad admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:09.594427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a558f5f-0d70-4fd2-98e0-e1e8ab244d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:09.595215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a558f5f-0d70-4fd2-98e0-e1e8ab244d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:09.595544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a558f5f-0d70-4fd2-98e0-e1e8ab244d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:09.595890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a558f5f-0d70-4fd2-98e0-e1e8ab244d9c admin admin] PATH_INFO: `/v3/projects/9cba124a13c44703a0625f8d18cb5037/users/3c8766948d6845cb89a7be69a8a88601/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:09.597876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a558f5f-0d70-4fd2-98e0-e1e8ab244d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cba124a13c44703a0625f8d18cb5037, user_id=3c8766948d6845cb89a7be69a8a88601, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:09.600568 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a558f5f-0d70-4fd2-98e0-e1e8ab244d9c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:09.615854 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1029/2073] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:09 2022] PUT /identity/v3/projects/9cba124a13c44703a0625f8d18cb5037/users/3c8766948d6845cb89a7be69a8a88601/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:10.086797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:11.137610 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e06d50-80fe-4b47-90e7-18b3f8b070bf admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:11.139422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-74e06d50-80fe-4b47-90e7-18b3f8b070bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:11.154678 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e06d50-80fe-4b47-90e7-18b3f8b070bf admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:11.217439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c62748-38f0-42e5-9f77-3c3bb30b5135 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:11.217439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c62748-38f0-42e5-9f77-3c3bb30b5135 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:11.217439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c62748-38f0-42e5-9f77-3c3bb30b5135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:11.246973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:11.248253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:11.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:11.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] PATH_INFO: `/v3/users/eafa589bad64499795ff5401ad26c8a8` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:11.251649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eafa589bad64499795ff5401ad26c8a8)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:11.253081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-a5c62748-38f0-42e5-9f77-3c3bb30b5135 None None] MFA Rules not processed for user `3c8766948d6845cb89a7be69a8a88601`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:11.253341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:11.254353 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a5c62748-38f0-42e5-9f77-3c3bb30b5135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:11.332034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1030/2074] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:28:11.345684 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c62748-38f0-42e5-9f77-3c3bb30b5135 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:11.405008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c62748-38f0-42e5-9f77-3c3bb30b5135 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:11.408660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03abc5-7da5-4b5c-ac0e-0fc52ec4a6e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:11.409273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03abc5-7da5-4b5c-ac0e-0fc52ec4a6e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:11.409556 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03abc5-7da5-4b5c-ac0e-0fc52ec4a6e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:11.409830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03abc5-7da5-4b5c-ac0e-0fc52ec4a6e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:11.489310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03abc5-7da5-4b5c-ac0e-0fc52ec4a6e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:11.493207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03abc5-7da5-4b5c-ac0e-0fc52ec4a6e9 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:11.505075 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1031/2075] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:11.566145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eafa589bad64499795ff5401ad26c8a8'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:11.574184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eafa589bad64499795ff5401ad26c8a8'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:11.657847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eafa589bad64499795ff5401ad26c8a8'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:11.658775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eafa589bad64499795ff5401ad26c8a8'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:11.671652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eafa589bad64499795ff5401ad26c8a8'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:11.677645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1045/2076] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:11 2022] DELETE /identity/v3/users/eafa589bad64499795ff5401ad26c8a8 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:11.682298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a5f77-210f-40a7-812b-6bb3e09866a0 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:11.682587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a5f77-210f-40a7-812b-6bb3e09866a0 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:11.682891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a5f77-210f-40a7-812b-6bb3e09866a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:11.701676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-740a5f77-210f-40a7-812b-6bb3e09866a0 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:11.703347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-740a5f77-210f-40a7-812b-6bb3e09866a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:11.766725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1032/2077] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:11.778240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:11.779771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:11.818755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:11.852541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6357b2-00ec-4e9b-9909-bc48db0f07e0 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:11.856399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8026d-494b-471d-a054-caadda085afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:11.857035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8026d-494b-471d-a054-caadda085afd service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:11.857311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8026d-494b-471d-a054-caadda085afd service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:11.857578 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8026d-494b-471d-a054-caadda085afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:11.866954 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8026d-494b-471d-a054-caadda085afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:11.868461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8026d-494b-471d-a054-caadda085afd service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:11.878625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1046/2078] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:12.269806 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:12.340047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a5f77-210f-40a7-812b-6bb3e09866a0 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:12.345340 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-740a5f77-210f-40a7-812b-6bb3e09866a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:12.369868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a5f77-210f-40a7-812b-6bb3e09866a0 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:12.380144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:12.381041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:12.381734 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:12.382014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] PATH_INFO: `/v3/projects/8cf35ea597fc452ab2999e230751a26a` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:12.383446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cf35ea597fc452ab2999e230751a26a)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:12.385914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:12.582364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cf35ea597fc452ab2999e230751a26a', 'request_id': 'req-7bebf84a-276e-4903-ab42-5d0d0bce4303'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:12.588959 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cf35ea597fc452ab2999e230751a26a', 'request_id': 'req-7bebf84a-276e-4903-ab42-5d0d0bce4303'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:12.714526 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cf35ea597fc452ab2999e230751a26a', 'request_id': 'req-7bebf84a-276e-4903-ab42-5d0d0bce4303'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:12.726967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] The token cache is being invalidate because project 8cf35ea597fc452ab2999e230751a26a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:12.727288 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:12.727951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:12.729476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1033/2079] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:12 2022] DELETE /identity/v3/projects/8cf35ea597fc452ab2999e230751a26a => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:12.734140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8026d-494b-471d-a054-caadda085afd service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:12.744677 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8026d-494b-471d-a054-caadda085afd service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:12.831756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c8e38-9f72-4024-a519-bfa98d0e6749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:12.832834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c8e38-9f72-4024-a519-bfa98d0e6749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:12.833630 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c8e38-9f72-4024-a519-bfa98d0e6749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:12.834193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c8e38-9f72-4024-a519-bfa98d0e6749 admin admin] PATH_INFO: `/v3/users/621c9c15b8ee4fbd8493c0f21dd57e21` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:12.836642 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:12.836860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c8e38-9f72-4024-a519-bfa98d0e6749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=621c9c15b8ee4fbd8493c0f21dd57e21)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:12.838449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:12.838855 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c8e38-9f72-4024-a519-bfa98d0e6749 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:12.852302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebf84a-276e-4903-ab42-5d0d0bce4303 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:12.857738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:12.858813 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:12.859036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:12.859320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] PATH_INFO: `/v3/users/c5019bb7e58b4661abc69ebe45f53b23` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:12.860512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5019bb7e58b4661abc69ebe45f53b23)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:12.862465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:13.135150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-882c8e38-9f72-4024-a519-bfa98d0e6749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '621c9c15b8ee4fbd8493c0f21dd57e21'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:13.135920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5019bb7e58b4661abc69ebe45f53b23'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:13.136601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5019bb7e58b4661abc69ebe45f53b23'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:13.142753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-882c8e38-9f72-4024-a519-bfa98d0e6749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '621c9c15b8ee4fbd8493c0f21dd57e21'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:13.145141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5019bb7e58b4661abc69ebe45f53b23'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:13.280382 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-882c8e38-9f72-4024-a519-bfa98d0e6749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '621c9c15b8ee4fbd8493c0f21dd57e21'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:13.281021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-882c8e38-9f72-4024-a519-bfa98d0e6749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '621c9c15b8ee4fbd8493c0f21dd57e21'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:13.293979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-882c8e38-9f72-4024-a519-bfa98d0e6749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '621c9c15b8ee4fbd8493c0f21dd57e21'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:13.299831 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1047/2080] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:12 2022] DELETE /identity/v3/users/621c9c15b8ee4fbd8493c0f21dd57e21 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:13.320125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5019bb7e58b4661abc69ebe45f53b23'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:13.320775 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5019bb7e58b4661abc69ebe45f53b23'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:13.340880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1034/2081] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:12 2022] DELETE /identity/v3/users/c5019bb7e58b4661abc69ebe45f53b23 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:13.345753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e5701-2d13-4a2d-bf96-3be95033d2a6 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:13.346049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e5701-2d13-4a2d-bf96-3be95033d2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:13.346371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e5701-2d13-4a2d-bf96-3be95033d2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:13.366468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-bd6e5701-2d13-4a2d-bf96-3be95033d2a6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:13.367698 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bd6e5701-2d13-4a2d-bf96-3be95033d2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:13.427898 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1048/2082] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:13.438507 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:13.440027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:13.480999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:13.501687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7031d-9bab-404d-b635-f9574d467f9f admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:13.505690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424c75a-906c-4b57-ba97-dc9fd9242063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:13.506315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424c75a-906c-4b57-ba97-dc9fd9242063 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:13.506586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424c75a-906c-4b57-ba97-dc9fd9242063 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:13.506861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424c75a-906c-4b57-ba97-dc9fd9242063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:13.524769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424c75a-906c-4b57-ba97-dc9fd9242063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:13.526642 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424c75a-906c-4b57-ba97-dc9fd9242063 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:13.539103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1035/2083] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:13.695933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:13.817865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e5701-2d13-4a2d-bf96-3be95033d2a6 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:13.819346 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bd6e5701-2d13-4a2d-bf96-3be95033d2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:13.833389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e5701-2d13-4a2d-bf96-3be95033d2a6 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:13.838602 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367919d-756a-42eb-a6ba-3133953d66c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:13.839311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367919d-756a-42eb-a6ba-3133953d66c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:13.839657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367919d-756a-42eb-a6ba-3133953d66c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:13.840050 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367919d-756a-42eb-a6ba-3133953d66c1 admin admin] PATH_INFO: `/v3/projects/c7a4412c22ea47ea90d48bc754986e71` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:13.841367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367919d-756a-42eb-a6ba-3133953d66c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7a4412c22ea47ea90d48bc754986e71)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:13.843959 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367919d-756a-42eb-a6ba-3133953d66c1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:13.921423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:14.001274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4367919d-756a-42eb-a6ba-3133953d66c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7a4412c22ea47ea90d48bc754986e71', 'request_id': 'req-4367919d-756a-42eb-a6ba-3133953d66c1'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.008134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4367919d-756a-42eb-a6ba-3133953d66c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7a4412c22ea47ea90d48bc754986e71', 'request_id': 'req-4367919d-756a-42eb-a6ba-3133953d66c1'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.015327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4367919d-756a-42eb-a6ba-3133953d66c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7a4412c22ea47ea90d48bc754986e71', 'request_id': 'req-4367919d-756a-42eb-a6ba-3133953d66c1'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.082208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424c75a-906c-4b57-ba97-dc9fd9242063 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:14.083674 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1424c75a-906c-4b57-ba97-dc9fd9242063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:14.098110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4367919d-756a-42eb-a6ba-3133953d66c1 admin admin] The token cache is being invalidate because project c7a4412c22ea47ea90d48bc754986e71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:14.098401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4367919d-756a-42eb-a6ba-3133953d66c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.104354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424c75a-906c-4b57-ba97-dc9fd9242063 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:14.104477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4367919d-756a-42eb-a6ba-3133953d66c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.104477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1049/2084] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:13 2022] DELETE /identity/v3/projects/c7a4412c22ea47ea90d48bc754986e71 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:14.108634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84e207-f5da-4081-8cc4-86bb07a4a103 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:14.108926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84e207-f5da-4081-8cc4-86bb07a4a103 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:14.109327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84e207-f5da-4081-8cc4-86bb07a4a103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:14.129998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-8a84e207-f5da-4081-8cc4-86bb07a4a103 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:14.131212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8a84e207-f5da-4081-8cc4-86bb07a4a103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:14.160085 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:14.160851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:14.161128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:14.161398 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] PATH_INFO: `/v3/projects/6432847b417645a087ff8e4ddc383898` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:14.162548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6432847b417645a087ff8e4ddc383898)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:14.164643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:14.192131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1050/2085] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:14.199566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab887568-5085-40c1-ba78-759c63300957 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:14.199876 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab887568-5085-40c1-ba78-759c63300957 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:14.200136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab887568-5085-40c1-ba78-759c63300957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:14.219916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ab887568-5085-40c1-ba78-759c63300957 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:14.221143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ab887568-5085-40c1-ba78-759c63300957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:14.229114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1051/2086] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:28:14.233628 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab887568-5085-40c1-ba78-759c63300957 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:14.242349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab887568-5085-40c1-ba78-759c63300957 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:14.249089 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b51b6-9ec0-45ed-8b8c-5641ff1590ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:14.249706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b51b6-9ec0-45ed-8b8c-5641ff1590ab admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:14.249992 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b51b6-9ec0-45ed-8b8c-5641ff1590ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:14.250264 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b51b6-9ec0-45ed-8b8c-5641ff1590ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:14.250960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b51b6-9ec0-45ed-8b8c-5641ff1590ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:14.251241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b51b6-9ec0-45ed-8b8c-5641ff1590ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:14.252970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b51b6-9ec0-45ed-8b8c-5641ff1590ab admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:14.260374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6432847b417645a087ff8e4ddc383898', 'request_id': 'req-9f77af17-c3f9-4156-84f4-4c739f40bdd4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.268457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1052/2087] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:28:14.268618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6432847b417645a087ff8e4ddc383898', 'request_id': 'req-9f77af17-c3f9-4156-84f4-4c739f40bdd4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.270360 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b51b6-9ec0-45ed-8b8c-5641ff1590ab admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:14.280914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b51b6-9ec0-45ed-8b8c-5641ff1590ab admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:14.326495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6432847b417645a087ff8e4ddc383898', 'request_id': 'req-9f77af17-c3f9-4156-84f4-4c739f40bdd4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.337434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066117b3-306d-47ff-ac80-33edd9c1cdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:14.338153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066117b3-306d-47ff-ac80-33edd9c1cdfe admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:14.338436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066117b3-306d-47ff-ac80-33edd9c1cdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:14.338707 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066117b3-306d-47ff-ac80-33edd9c1cdfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:14.338800 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] The token cache is being invalidate because project 6432847b417645a087ff8e4ddc383898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:14.339019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.339330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066117b3-306d-47ff-ac80-33edd9c1cdfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:14.339645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.341315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066117b3-306d-47ff-ac80-33edd9c1cdfe admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:14.347007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1036/2088] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:14 2022] DELETE /identity/v3/projects/6432847b417645a087ff8e4ddc383898 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:14.347946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:14.350410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:14.362863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77af17-c3f9-4156-84f4-4c739f40bdd4 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:14.367906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467a829-7d32-41a2-a14e-657d17263bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:14.375465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467a829-7d32-41a2-a14e-657d17263bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:14.375465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467a829-7d32-41a2-a14e-657d17263bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:14.375465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467a829-7d32-41a2-a14e-657d17263bb0 admin admin] PATH_INFO: `/v3/users/fd5d45e519f8454f9823e35c3e0e871f` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:14.375465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467a829-7d32-41a2-a14e-657d17263bb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5d45e519f8454f9823e35c3e0e871f)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:14.375465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467a829-7d32-41a2-a14e-657d17263bb0 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:14.442594 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1053/2089] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:14 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:14.446362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066117b3-306d-47ff-ac80-33edd9c1cdfe admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:14.448023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-066117b3-306d-47ff-ac80-33edd9c1cdfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:14.512937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066117b3-306d-47ff-ac80-33edd9c1cdfe admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:14.518348 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9f09a-8c47-4f85-9066-4879073f540e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:14.518953 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9f09a-8c47-4f85-9066-4879073f540e admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:14.519229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9f09a-8c47-4f85-9066-4879073f540e admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:14.519501 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9f09a-8c47-4f85-9066-4879073f540e admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:14.520130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9f09a-8c47-4f85-9066-4879073f540e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:14.522072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9f09a-8c47-4f85-9066-4879073f540e admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:14.596300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e467a829-7d32-41a2-a14e-657d17263bb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5d45e519f8454f9823e35c3e0e871f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.596959 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e467a829-7d32-41a2-a14e-657d17263bb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5d45e519f8454f9823e35c3e0e871f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.606967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e467a829-7d32-41a2-a14e-657d17263bb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5d45e519f8454f9823e35c3e0e871f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.676919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1054/2090] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:14 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:14.682224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9f09a-8c47-4f85-9066-4879073f540e admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:14.691251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9f09a-8c47-4f85-9066-4879073f540e admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:14.695888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c7d5a-aca6-4e8b-aa18-119ec5ba4888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:14.696447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c7d5a-aca6-4e8b-aa18-119ec5ba4888 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:14.696727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c7d5a-aca6-4e8b-aa18-119ec5ba4888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:14.697001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c7d5a-aca6-4e8b-aa18-119ec5ba4888 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:14.697639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c7d5a-aca6-4e8b-aa18-119ec5ba4888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:14.699340 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c7d5a-aca6-4e8b-aa18-119ec5ba4888 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:14.709652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1055/2091] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:14.713477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c7d5a-aca6-4e8b-aa18-119ec5ba4888 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:14.725081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e467a829-7d32-41a2-a14e-657d17263bb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5d45e519f8454f9823e35c3e0e871f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.727709 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e467a829-7d32-41a2-a14e-657d17263bb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5d45e519f8454f9823e35c3e0e871f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:14.727827 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c7d5a-aca6-4e8b-aa18-119ec5ba4888 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:14.733898 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357c882-9c38-47a3-b5e4-23de7768d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:14.734535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357c882-9c38-47a3-b5e4-23de7768d9ec admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:14.734795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357c882-9c38-47a3-b5e4-23de7768d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:14.735066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357c882-9c38-47a3-b5e4-23de7768d9ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:14.735843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357c882-9c38-47a3-b5e4-23de7768d9ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:14.737500 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357c882-9c38-47a3-b5e4-23de7768d9ec admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:14.759214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1056/2092] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:28:14.759214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357c882-9c38-47a3-b5e4-23de7768d9ec admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:14.759214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3357c882-9c38-47a3-b5e4-23de7768d9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:14.759586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1037/2093] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:14 2022] DELETE /identity/v3/users/fd5d45e519f8454f9823e35c3e0e871f => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:14.770813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357c882-9c38-47a3-b5e4-23de7768d9ec admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:14.850444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc58ec-98f1-4455-bf0d-4495f2618823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:14.852005 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc58ec-98f1-4455-bf0d-4495f2618823 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:14.852341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc58ec-98f1-4455-bf0d-4495f2618823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:14.852622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc58ec-98f1-4455-bf0d-4495f2618823 admin admin] PATH_INFO: `/v3/projects/df00ebbd06ad4f1dbf9d96e099a7ed11/users/b2f3d193a4b945e3b054adb4c43551c8/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:14.871056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc58ec-98f1-4455-bf0d-4495f2618823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df00ebbd06ad4f1dbf9d96e099a7ed11, user_id=b2f3d193a4b945e3b054adb4c43551c8, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:14.875182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:14.875343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc58ec-98f1-4455-bf0d-4495f2618823 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:14.995398 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1057/2094] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:14 2022] PUT /identity/v3/projects/df00ebbd06ad4f1dbf9d96e099a7ed11/users/b2f3d193a4b945e3b054adb4c43551c8/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:15.006014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfeede-c92a-4256-8b7f-718bc2683152 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:15.006433 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfeede-c92a-4256-8b7f-718bc2683152 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:15.006909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfeede-c92a-4256-8b7f-718bc2683152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:15.041441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-74bfeede-c92a-4256-8b7f-718bc2683152 None None] MFA Rules not processed for user `b2f3d193a4b945e3b054adb4c43551c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:15.042688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-74bfeede-c92a-4256-8b7f-718bc2683152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:15.118614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1038/2095] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:15.123218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc58ec-98f1-4455-bf0d-4495f2618823 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:15.186644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc58ec-98f1-4455-bf0d-4495f2618823 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:15.191825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f18cff-c7a9-475f-9fdd-73fcf5bea841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:15.192465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f18cff-c7a9-475f-9fdd-73fcf5bea841 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:15.192747 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f18cff-c7a9-475f-9fdd-73fcf5bea841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:15.193047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f18cff-c7a9-475f-9fdd-73fcf5bea841 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:15.193676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f18cff-c7a9-475f-9fdd-73fcf5bea841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:15.195700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f18cff-c7a9-475f-9fdd-73fcf5bea841 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:15.235142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:15.310686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1058/2096] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:15 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:15.314582 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfeede-c92a-4256-8b7f-718bc2683152 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:15.325796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfeede-c92a-4256-8b7f-718bc2683152 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:15.347489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f18cff-c7a9-475f-9fdd-73fcf5bea841 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:15.349756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b1f18cff-c7a9-475f-9fdd-73fcf5bea841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:15.384792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a98a52-5625-454e-98df-f96a60c648a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:15.385502 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a98a52-5625-454e-98df-f96a60c648a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:15.385835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a98a52-5625-454e-98df-f96a60c648a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:15.386095 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a98a52-5625-454e-98df-f96a60c648a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:15.387243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a98a52-5625-454e-98df-f96a60c648a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:15.389215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a98a52-5625-454e-98df-f96a60c648a2 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:15.422911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f18cff-c7a9-475f-9fdd-73fcf5bea841 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:15.427647 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:15.428303 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:15.428622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:15.428861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] PATH_INFO: `/v3/projects/934517c69faf40d9a402ec644849e1d9` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:15.429975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=934517c69faf40d9a402ec644849e1d9)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:15.431881 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:15.505809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1039/2097] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:15 2022] POST /identity/v3/users => generated 299 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:15.507052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a98a52-5625-454e-98df-f96a60c648a2 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:15.520044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a98a52-5625-454e-98df-f96a60c648a2 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:15.522967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1b093-aa78-4400-b9ea-c370dafaf988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:15.523564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1b093-aa78-4400-b9ea-c370dafaf988 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:15.523848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1b093-aa78-4400-b9ea-c370dafaf988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:15.524199 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1b093-aa78-4400-b9ea-c370dafaf988 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:15.524911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1b093-aa78-4400-b9ea-c370dafaf988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:15.526891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1b093-aa78-4400-b9ea-c370dafaf988 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:15.539550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1040/2098] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:28:15.539550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1b093-aa78-4400-b9ea-c370dafaf988 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:15.540276 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-04b1b093-aa78-4400-b9ea-c370dafaf988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:15.595879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1b093-aa78-4400-b9ea-c370dafaf988 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:15.606924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '934517c69faf40d9a402ec644849e1d9', 'request_id': 'req-91dd4efc-6325-43be-913f-a29ae24e7480'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:15.616269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '934517c69faf40d9a402ec644849e1d9', 'request_id': 'req-91dd4efc-6325-43be-913f-a29ae24e7480'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:15.622836 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '934517c69faf40d9a402ec644849e1d9', 'request_id': 'req-91dd4efc-6325-43be-913f-a29ae24e7480'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:15.656328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6db1f-66b6-41a9-bc8a-da534e0cdb49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:15.657087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6db1f-66b6-41a9-bc8a-da534e0cdb49 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:15.657372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6db1f-66b6-41a9-bc8a-da534e0cdb49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:15.657644 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6db1f-66b6-41a9-bc8a-da534e0cdb49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:15.659445 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-91c6db1f-66b6-41a9-bc8a-da534e0cdb49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:15.719998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] The token cache is being invalidate because project 934517c69faf40d9a402ec644849e1d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:15.720311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:15.720973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:15.722123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6db1f-66b6-41a9-bc8a-da534e0cdb49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:15.722551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1059/2099] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:15 2022] DELETE /identity/v3/projects/934517c69faf40d9a402ec644849e1d9 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:15.723557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:15.724852 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:15.725242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6db1f-66b6-41a9-bc8a-da534e0cdb49 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:15.734951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1041/2100] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:15.736262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ac460-b0a9-4802-8e88-8e13ce9a7271 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:15.736615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ac460-b0a9-4802-8e88-8e13ce9a7271 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:15.736957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ac460-b0a9-4802-8e88-8e13ce9a7271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:15.758164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-2b1ac460-b0a9-4802-8e88-8e13ce9a7271 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:15.759419 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2b1ac460-b0a9-4802-8e88-8e13ce9a7271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:15.791627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd4efc-6325-43be-913f-a29ae24e7480 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:15.796574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b102e-57d7-431b-90eb-2a4b578a7f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:15.797328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b102e-57d7-431b-90eb-2a4b578a7f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:15.799355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b102e-57d7-431b-90eb-2a4b578a7f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:15.799355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b102e-57d7-431b-90eb-2a4b578a7f96 admin admin] PATH_INFO: `/v3/projects/87fa343cc5ce4697b0127fb00eae792e/users/4324d9ea4c72407db8b27bcf8892c0f1/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:15.814649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b102e-57d7-431b-90eb-2a4b578a7f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87fa343cc5ce4697b0127fb00eae792e, user_id=4324d9ea4c72407db8b27bcf8892c0f1, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:15.817590 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b102e-57d7-431b-90eb-2a4b578a7f96 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:15.827792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1042/2101] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:15.835987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144090fb-d6cb-4e0b-8c9d-8d1d220b258c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:15.836419 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144090fb-d6cb-4e0b-8c9d-8d1d220b258c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:15.836756 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144090fb-d6cb-4e0b-8c9d-8d1d220b258c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:15.857285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-144090fb-d6cb-4e0b-8c9d-8d1d220b258c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:15.858707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-144090fb-d6cb-4e0b-8c9d-8d1d220b258c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:15.869926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1043/2102] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:15.874595 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144090fb-d6cb-4e0b-8c9d-8d1d220b258c None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:15.884362 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144090fb-d6cb-4e0b-8c9d-8d1d220b258c None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:15.892438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476e93c-0b66-4807-8460-5d097e7ffca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:15.896606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476e93c-0b66-4807-8460-5d097e7ffca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:15.896606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476e93c-0b66-4807-8460-5d097e7ffca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:15.896606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476e93c-0b66-4807-8460-5d097e7ffca0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:15.896606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476e93c-0b66-4807-8460-5d097e7ffca0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:15.896606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476e93c-0b66-4807-8460-5d097e7ffca0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:15.896606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476e93c-0b66-4807-8460-5d097e7ffca0 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:15.912043 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1044/2103] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:28:15.924841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476e93c-0b66-4807-8460-5d097e7ffca0 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:15.943073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476e93c-0b66-4807-8460-5d097e7ffca0 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:15.950559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38be651-4128-41ba-b2ac-f037a70bde83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:15.951249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38be651-4128-41ba-b2ac-f037a70bde83 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:15.951711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38be651-4128-41ba-b2ac-f037a70bde83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:15.952295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38be651-4128-41ba-b2ac-f037a70bde83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:15.952968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38be651-4128-41ba-b2ac-f037a70bde83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:15.955015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38be651-4128-41ba-b2ac-f037a70bde83 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:15.980512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1060/2104] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:15 2022] PUT /identity/v3/projects/87fa343cc5ce4697b0127fb00eae792e/users/4324d9ea4c72407db8b27bcf8892c0f1/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:15.980512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b102e-57d7-431b-90eb-2a4b578a7f96 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:15.980512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d02b102e-57d7-431b-90eb-2a4b578a7f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:16.044479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b102e-57d7-431b-90eb-2a4b578a7f96 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:16.049033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b6d40-f677-4f66-9657-206f0596471a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:16.049675 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b6d40-f677-4f66-9657-206f0596471a service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:16.049957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b6d40-f677-4f66-9657-206f0596471a service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:16.050232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b6d40-f677-4f66-9657-206f0596471a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:16.051976 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-fa3b6d40-f677-4f66-9657-206f0596471a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:16.119050 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b6d40-f677-4f66-9657-206f0596471a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:16.120435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b6d40-f677-4f66-9657-206f0596471a service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:16.130829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1061/2105] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:16.131744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b6d40-f677-4f66-9657-206f0596471a service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:16.137888 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1045/2106] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:15 2022] POST /identity/v3/projects => generated 365 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:16.138873 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38be651-4128-41ba-b2ac-f037a70bde83 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:16.195146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b6d40-f677-4f66-9657-206f0596471a service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:16.200126 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39716a19-9e7e-46f9-9096-41f26d6b187e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:16.200915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39716a19-9e7e-46f9-9096-41f26d6b187e admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:16.201201 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39716a19-9e7e-46f9-9096-41f26d6b187e admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:16.201688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39716a19-9e7e-46f9-9096-41f26d6b187e admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:16.202142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39716a19-9e7e-46f9-9096-41f26d6b187e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:16.203859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39716a19-9e7e-46f9-9096-41f26d6b187e admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:16.204262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38be651-4128-41ba-b2ac-f037a70bde83 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:16.209421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f452e0-2a61-4b79-8994-e71f4889640b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:16.210176 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f452e0-2a61-4b79-8994-e71f4889640b admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:16.210548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f452e0-2a61-4b79-8994-e71f4889640b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:16.211073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f452e0-2a61-4b79-8994-e71f4889640b admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:16.212700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f452e0-2a61-4b79-8994-e71f4889640b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:16.214213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f452e0-2a61-4b79-8994-e71f4889640b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:16.218191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1062/2107] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:16.219113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39716a19-9e7e-46f9-9096-41f26d6b187e admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:16.229778 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39716a19-9e7e-46f9-9096-41f26d6b187e admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:16.235353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22026585-5ed4-42c1-942c-3062cfde12be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:16.236065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22026585-5ed4-42c1-942c-3062cfde12be admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:16.236383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22026585-5ed4-42c1-942c-3062cfde12be admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:16.236757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22026585-5ed4-42c1-942c-3062cfde12be admin admin] PATH_INFO: `/v3/projects/87fa343cc5ce4697b0127fb00eae792e/users/4324d9ea4c72407db8b27bcf8892c0f1/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:16.238807 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22026585-5ed4-42c1-942c-3062cfde12be admin admin] RBAC: Authorizing `identity:create_grant(project_id=87fa343cc5ce4697b0127fb00eae792e, user_id=4324d9ea4c72407db8b27bcf8892c0f1, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:16.241548 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22026585-5ed4-42c1-942c-3062cfde12be admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:16.257715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1063/2108] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:16 2022] PUT /identity/v3/projects/87fa343cc5ce4697b0127fb00eae792e/users/4324d9ea4c72407db8b27bcf8892c0f1/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:16.270499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a0a34-74f1-47a2-b3f1-3b90bbce4c26 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:16.270797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a0a34-74f1-47a2-b3f1-3b90bbce4c26 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:16.271074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a0a34-74f1-47a2-b3f1-3b90bbce4c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:16.310357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-5d9a0a34-74f1-47a2-b3f1-3b90bbce4c26 None None] MFA Rules not processed for user `4324d9ea4c72407db8b27bcf8892c0f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:16.312034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5d9a0a34-74f1-47a2-b3f1-3b90bbce4c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:16.389972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1046/2109] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:16 2022] POST /identity/v3/users => generated 283 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:16.394322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f452e0-2a61-4b79-8994-e71f4889640b admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:16.415772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f452e0-2a61-4b79-8994-e71f4889640b admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:16.448349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1064/2110] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:16.510013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae8fc1-acac-49c0-bdff-4c1edea361b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:16.510624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae8fc1-acac-49c0-bdff-4c1edea361b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:16.515262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae8fc1-acac-49c0-bdff-4c1edea361b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:16.515262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae8fc1-acac-49c0-bdff-4c1edea361b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:16.515262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae8fc1-acac-49c0-bdff-4c1edea361b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:16.515262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae8fc1-acac-49c0-bdff-4c1edea361b2 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:16.523494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1047/2111] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:16.528236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a0a34-74f1-47a2-b3f1-3b90bbce4c26 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:16.538746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a0a34-74f1-47a2-b3f1-3b90bbce4c26 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:16.544002 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ad540-1357-4913-8640-2e53bcf7c795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:16.544627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ad540-1357-4913-8640-2e53bcf7c795 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:16.549810 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ad540-1357-4913-8640-2e53bcf7c795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:16.549810 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ad540-1357-4913-8640-2e53bcf7c795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:16.549810 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ad540-1357-4913-8640-2e53bcf7c795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:16.549810 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ad540-1357-4913-8640-2e53bcf7c795 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:16.557213 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1065/2112] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:16.561300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae8fc1-acac-49c0-bdff-4c1edea361b2 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:16.571184 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae8fc1-acac-49c0-bdff-4c1edea361b2 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:16.576197 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400c17e-db5d-4643-83f4-7e00b03f7d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:16.576855 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400c17e-db5d-4643-83f4-7e00b03f7d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:16.577138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400c17e-db5d-4643-83f4-7e00b03f7d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:16.577415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400c17e-db5d-4643-83f4-7e00b03f7d79 admin admin] PATH_INFO: `/v3/projects/c0f6de985fec4a64bfb79f755e9d4856/users/cd4c1cd9fdc546f486138259bd9c4e4e/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:16.593631 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400c17e-db5d-4643-83f4-7e00b03f7d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0f6de985fec4a64bfb79f755e9d4856, user_id=cd4c1cd9fdc546f486138259bd9c4e4e, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:16.596326 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400c17e-db5d-4643-83f4-7e00b03f7d79 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:16.756507 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1048/2113] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:16 2022] PUT /identity/v3/projects/c0f6de985fec4a64bfb79f755e9d4856/users/cd4c1cd9fdc546f486138259bd9c4e4e/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:16.765080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ad540-1357-4913-8640-2e53bcf7c795 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:16.767890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf89d6-984a-414d-805c-375167a20399 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:16.768435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf89d6-984a-414d-805c-375167a20399 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:16.768736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf89d6-984a-414d-805c-375167a20399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:16.798687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-05bf89d6-984a-414d-805c-375167a20399 None None] MFA Rules not processed for user `cd4c1cd9fdc546f486138259bd9c4e4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:16.799898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-05bf89d6-984a-414d-805c-375167a20399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:16.822670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ad540-1357-4913-8640-2e53bcf7c795 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:16.882535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950b15e-2d10-467b-b6be-537f366935c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:16.882852 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1049/2114] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:16.883579 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950b15e-2d10-467b-b6be-537f366935c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:16.883864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950b15e-2d10-467b-b6be-537f366935c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:16.884137 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950b15e-2d10-467b-b6be-537f366935c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:16.887429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf89d6-984a-414d-805c-375167a20399 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:16.967153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf89d6-984a-414d-805c-375167a20399 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:16.972074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda424-d047-42de-8aea-34c5da369c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:16.972733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda424-d047-42de-8aea-34c5da369c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:16.973019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda424-d047-42de-8aea-34c5da369c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:16.973307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda424-d047-42de-8aea-34c5da369c32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:16.973933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eda424-d047-42de-8aea-34c5da369c32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:16.975126 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950b15e-2d10-467b-b6be-537f366935c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:16.975850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eda424-d047-42de-8aea-34c5da369c32 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:16.976697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950b15e-2d10-467b-b6be-537f366935c5 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:16.987453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1066/2115] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:28:17.021458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950b15e-2d10-467b-b6be-537f366935c5 service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:17.033912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950b15e-2d10-467b-b6be-537f366935c5 service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:17.040556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea7409-96ba-4188-bbd3-4b3453f80229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:17.041436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea7409-96ba-4188-bbd3-4b3453f80229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:17.041785 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea7409-96ba-4188-bbd3-4b3453f80229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:17.042134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea7409-96ba-4188-bbd3-4b3453f80229 admin admin] PATH_INFO: `/v3/users/b2f3d193a4b945e3b054adb4c43551c8` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:17.043406 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea7409-96ba-4188-bbd3-4b3453f80229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2f3d193a4b945e3b054adb4c43551c8)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:17.045577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea7409-96ba-4188-bbd3-4b3453f80229 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:17.113708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1050/2116] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:16 2022] POST /identity/v3/projects => generated 367 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:17.117305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda424-d047-42de-8aea-34c5da369c32 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:17.194046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda424-d047-42de-8aea-34c5da369c32 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:17.200126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9bde6-a338-4244-abb0-6726cd9052bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:17.200767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9bde6-a338-4244-abb0-6726cd9052bc admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:17.201047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9bde6-a338-4244-abb0-6726cd9052bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:17.201322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9bde6-a338-4244-abb0-6726cd9052bc admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:17.202051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9bde6-a338-4244-abb0-6726cd9052bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:17.203913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9bde6-a338-4244-abb0-6726cd9052bc admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:17.349377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1051/2117] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:17 2022] POST /identity/v3/users => generated 290 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:17.358126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9bde6-a338-4244-abb0-6726cd9052bc admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:17.365321 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9bde6-a338-4244-abb0-6726cd9052bc admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:17.370108 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025e6c3-2b99-457b-a42b-78f1c6466bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:17.370885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025e6c3-2b99-457b-a42b-78f1c6466bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:17.371985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025e6c3-2b99-457b-a42b-78f1c6466bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:17.372222 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025e6c3-2b99-457b-a42b-78f1c6466bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:17.372893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e025e6c3-2b99-457b-a42b-78f1c6466bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:17.374584 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e025e6c3-2b99-457b-a42b-78f1c6466bd6 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:17.384979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1052/2118] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:17.389741 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025e6c3-2b99-457b-a42b-78f1c6466bd6 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:17.390027 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e4ea7409-96ba-4188-bbd3-4b3453f80229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2f3d193a4b945e3b054adb4c43551c8'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:17.397934 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e4ea7409-96ba-4188-bbd3-4b3453f80229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2f3d193a4b945e3b054adb4c43551c8'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:17.399911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025e6c3-2b99-457b-a42b-78f1c6466bd6 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:17.405418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2b037-5c8d-4b30-8da2-460ea19a1d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:17.406072 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2b037-5c8d-4b30-8da2-460ea19a1d3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:17.406356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2b037-5c8d-4b30-8da2-460ea19a1d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:17.406623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2b037-5c8d-4b30-8da2-460ea19a1d3c admin admin] PATH_INFO: `/v3/projects/5c0c01ccd4c849788b82ec4d99899047/users/da4d398460344216ac9aed36cd3efbff/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:17.421408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2b037-5c8d-4b30-8da2-460ea19a1d3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0c01ccd4c849788b82ec4d99899047, user_id=da4d398460344216ac9aed36cd3efbff, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:17.424122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2b037-5c8d-4b30-8da2-460ea19a1d3c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:17.474721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:17.474858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e4ea7409-96ba-4188-bbd3-4b3453f80229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2f3d193a4b945e3b054adb4c43551c8'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:17.475518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e4ea7409-96ba-4188-bbd3-4b3453f80229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2f3d193a4b945e3b054adb4c43551c8'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:17.488120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e4ea7409-96ba-4188-bbd3-4b3453f80229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2f3d193a4b945e3b054adb4c43551c8'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:17.494782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1067/2119] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:17 2022] DELETE /identity/v3/users/b2f3d193a4b945e3b054adb4c43551c8 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:17.499660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626d67b-f206-44e6-88c5-f90ae9214bea None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:17.500018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626d67b-f206-44e6-88c5-f90ae9214bea None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:17.500368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626d67b-f206-44e6-88c5-f90ae9214bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:17.524422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-6626d67b-f206-44e6-88c5-f90ae9214bea None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:17.525646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-6626d67b-f206-44e6-88c5-f90ae9214bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:17.543465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1053/2120] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:17 2022] PUT /identity/v3/projects/5c0c01ccd4c849788b82ec4d99899047/users/da4d398460344216ac9aed36cd3efbff/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:17.552409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2b037-5c8d-4b30-8da2-460ea19a1d3c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:17.554633 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-19c2b037-5c8d-4b30-8da2-460ea19a1d3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:17.591125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1068/2121] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:28:17.601434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626d67b-f206-44e6-88c5-f90ae9214bea None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:17.611894 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-6626d67b-f206-44e6-88c5-f90ae9214bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:17.636422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2b037-5c8d-4b30-8da2-460ea19a1d3c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:17.642509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db031ec-28db-46e0-860e-5507f720a5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:17.643625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db031ec-28db-46e0-860e-5507f720a5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:17.643971 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db031ec-28db-46e0-860e-5507f720a5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:17.644350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db031ec-28db-46e0-860e-5507f720a5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:17.645183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db031ec-28db-46e0-860e-5507f720a5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:17.647010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db031ec-28db-46e0-860e-5507f720a5f3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:17.669028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1054/2122] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:17.669028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db031ec-28db-46e0-860e-5507f720a5f3 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:17.684320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db031ec-28db-46e0-860e-5507f720a5f3 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:17.689396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626d67b-f206-44e6-88c5-f90ae9214bea None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:17.690034 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a98dff-3005-4422-a996-a12d0c76038c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:17.691063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a98dff-3005-4422-a996-a12d0c76038c admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:17.691597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a98dff-3005-4422-a996-a12d0c76038c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:17.692046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a98dff-3005-4422-a996-a12d0c76038c admin admin] PATH_INFO: `/v3/projects/5c0c01ccd4c849788b82ec4d99899047/users/da4d398460344216ac9aed36cd3efbff/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:17.694527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a98dff-3005-4422-a996-a12d0c76038c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0c01ccd4c849788b82ec4d99899047, user_id=da4d398460344216ac9aed36cd3efbff, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:17.697983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a98dff-3005-4422-a996-a12d0c76038c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:17.719445 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1055/2123] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:17 2022] PUT /identity/v3/projects/5c0c01ccd4c849788b82ec4d99899047/users/da4d398460344216ac9aed36cd3efbff/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:17.721650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f04eca-997d-4321-9f88-a11d064498cf None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:17.722016 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f04eca-997d-4321-9f88-a11d064498cf None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:17.722364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f04eca-997d-4321-9f88-a11d064498cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:17.753866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-c3f04eca-997d-4321-9f88-a11d064498cf None None] MFA Rules not processed for user `da4d398460344216ac9aed36cd3efbff`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:17.755260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c3f04eca-997d-4321-9f88-a11d064498cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:17.763292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b21dee-a068-4269-801b-198b94c58af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:17.763915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b21dee-a068-4269-801b-198b94c58af2 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:17.764236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b21dee-a068-4269-801b-198b94c58af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:17.764550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b21dee-a068-4269-801b-198b94c58af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:17.778934 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b21dee-a068-4269-801b-198b94c58af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:17.780389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b21dee-a068-4269-801b-198b94c58af2 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:17.791458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1069/2124] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:28:17.843520 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1056/2125] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:28:17.863168 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b21dee-a068-4269-801b-198b94c58af2 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:17.864844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-39b21dee-a068-4269-801b-198b94c58af2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:17.928318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b21dee-a068-4269-801b-198b94c58af2 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:17.980586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4ef43-d9ef-4f34-999f-c117a716185b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:17.981317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4ef43-d9ef-4f34-999f-c117a716185b service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:17.981616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4ef43-d9ef-4f34-999f-c117a716185b service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:17.981896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4ef43-d9ef-4f34-999f-c117a716185b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:18.002315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4ef43-d9ef-4f34-999f-c117a716185b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:18.003980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4ef43-d9ef-4f34-999f-c117a716185b service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:18.016328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1070/2126] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:18.287743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f04eca-997d-4321-9f88-a11d064498cf None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:18.289186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c3f04eca-997d-4321-9f88-a11d064498cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:18.349984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:18.355266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f04eca-997d-4321-9f88-a11d064498cf None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:18.365074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:18.365779 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:18.366068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:18.366347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] PATH_INFO: `/v3/projects/df00ebbd06ad4f1dbf9d96e099a7ed11` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:18.367496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df00ebbd06ad4f1dbf9d96e099a7ed11)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:18.369639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:18.476420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df00ebbd06ad4f1dbf9d96e099a7ed11', 'request_id': 'req-a11cdabb-5f37-4004-9534-17504326bb00'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:18.481751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4ef43-d9ef-4f34-999f-c117a716185b service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:18.483213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df00ebbd06ad4f1dbf9d96e099a7ed11', 'request_id': 'req-a11cdabb-5f37-4004-9534-17504326bb00'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:18.490921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4ef43-d9ef-4f34-999f-c117a716185b service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:18.540157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111978ce-8b4a-42da-9215-470a3cd29094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:18.540798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111978ce-8b4a-42da-9215-470a3cd29094 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:18.541081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111978ce-8b4a-42da-9215-470a3cd29094 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:18.541511 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111978ce-8b4a-42da-9215-470a3cd29094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:18.543885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-111978ce-8b4a-42da-9215-470a3cd29094 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:18.583004 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df00ebbd06ad4f1dbf9d96e099a7ed11', 'request_id': 'req-a11cdabb-5f37-4004-9534-17504326bb00'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:18.594699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] The token cache is being invalidate because project df00ebbd06ad4f1dbf9d96e099a7ed11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:18.594992 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:18.595617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:18.597143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1057/2127] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:18 2022] DELETE /identity/v3/projects/df00ebbd06ad4f1dbf9d96e099a7ed11 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:18.600918 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:18.603139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111978ce-8b4a-42da-9215-470a3cd29094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:18.603252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:18.603338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111978ce-8b4a-42da-9215-470a3cd29094 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:18.617412 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1071/2128] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:28:18.706852 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11cdabb-5f37-4004-9534-17504326bb00 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:18.716711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bcfea-8e06-4740-8868-0b6869685ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:18.717369 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bcfea-8e06-4740-8868-0b6869685ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:18.717647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bcfea-8e06-4740-8868-0b6869685ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:18.717921 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bcfea-8e06-4740-8868-0b6869685ad7 admin admin] PATH_INFO: `/v3/users/4324d9ea4c72407db8b27bcf8892c0f1` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:18.719498 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bcfea-8e06-4740-8868-0b6869685ad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4324d9ea4c72407db8b27bcf8892c0f1)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:18.721466 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bcfea-8e06-4740-8868-0b6869685ad7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:18.991079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-352bcfea-8e06-4740-8868-0b6869685ad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4324d9ea4c72407db8b27bcf8892c0f1'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:18.991733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-352bcfea-8e06-4740-8868-0b6869685ad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4324d9ea4c72407db8b27bcf8892c0f1'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:18.999212 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-352bcfea-8e06-4740-8868-0b6869685ad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4324d9ea4c72407db8b27bcf8892c0f1'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:19.035221 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:19.195990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-352bcfea-8e06-4740-8868-0b6869685ad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4324d9ea4c72407db8b27bcf8892c0f1'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:19.196682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-352bcfea-8e06-4740-8868-0b6869685ad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4324d9ea4c72407db8b27bcf8892c0f1'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:19.214244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1058/2129] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:18 2022] DELETE /identity/v3/users/4324d9ea4c72407db8b27bcf8892c0f1 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:19.779140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111978ce-8b4a-42da-9215-470a3cd29094 service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:19.791505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111978ce-8b4a-42da-9215-470a3cd29094 service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:19.867431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:19.868376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:19.872790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:19.872790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd admin admin] PATH_INFO: `/v3/projects/87fa343cc5ce4697b0127fb00eae792e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:19.872790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=87fa343cc5ce4697b0127fb00eae792e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:19.875277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:20.111146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87fa343cc5ce4697b0127fb00eae792e', 'request_id': 'req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:20.117530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87fa343cc5ce4697b0127fb00eae792e', 'request_id': 'req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:20.124052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87fa343cc5ce4697b0127fb00eae792e', 'request_id': 'req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:20.245601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd admin admin] The token cache is being invalidate because project 87fa343cc5ce4697b0127fb00eae792e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:20.245821 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:20.246448 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9f8ceb1d-c700-4a35-b5d3-b6e2e79e62cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:20.247931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1072/2130] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:19 2022] DELETE /identity/v3/projects/87fa343cc5ce4697b0127fb00eae792e => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:20.253650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e0dc9-3c17-45f7-9c3e-aeaff922dabb None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:20.254114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e0dc9-3c17-45f7-9c3e-aeaff922dabb None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:20.254242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e0dc9-3c17-45f7-9c3e-aeaff922dabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:20.274336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-b24e0dc9-3c17-45f7-9c3e-aeaff922dabb None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:20.275535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b24e0dc9-3c17-45f7-9c3e-aeaff922dabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:20.338170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1059/2131] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:20.346446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564bc84-b213-4349-9ba8-501c233fc3d8 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:20.346733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564bc84-b213-4349-9ba8-501c233fc3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:20.347009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564bc84-b213-4349-9ba8-501c233fc3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:20.367028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d564bc84-b213-4349-9ba8-501c233fc3d8 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:20.368259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d564bc84-b213-4349-9ba8-501c233fc3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:20.376783 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1073/2132] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:20.381036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e0dc9-3c17-45f7-9c3e-aeaff922dabb None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:20.391150 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e0dc9-3c17-45f7-9c3e-aeaff922dabb None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:20.395997 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80e837-9569-4f60-8ee8-00a31c50adc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:20.396685 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80e837-9569-4f60-8ee8-00a31c50adc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:20.396964 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80e837-9569-4f60-8ee8-00a31c50adc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:20.397235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80e837-9569-4f60-8ee8-00a31c50adc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:20.397880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80e837-9569-4f60-8ee8-00a31c50adc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:20.398180 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80e837-9569-4f60-8ee8-00a31c50adc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:20.399812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80e837-9569-4f60-8ee8-00a31c50adc9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:20.411883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1060/2133] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:28:20.416067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564bc84-b213-4349-9ba8-501c233fc3d8 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:20.425333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564bc84-b213-4349-9ba8-501c233fc3d8 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:20.430062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180b9a8-edbd-4726-a8d9-1d6b61974727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:20.430688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180b9a8-edbd-4726-a8d9-1d6b61974727 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:20.430967 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180b9a8-edbd-4726-a8d9-1d6b61974727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:20.431241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180b9a8-edbd-4726-a8d9-1d6b61974727 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:20.431875 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180b9a8-edbd-4726-a8d9-1d6b61974727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:20.434327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180b9a8-edbd-4726-a8d9-1d6b61974727 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:20.533880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1074/2134] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:20 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:20.537819 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80e837-9569-4f60-8ee8-00a31c50adc9 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:20.548319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80e837-9569-4f60-8ee8-00a31c50adc9 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:20.554573 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:20.603202 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53a3ec-0c66-40f7-8704-35aeff6351e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:20.603806 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53a3ec-0c66-40f7-8704-35aeff6351e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:20.604085 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53a3ec-0c66-40f7-8704-35aeff6351e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:20.604390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53a3ec-0c66-40f7-8704-35aeff6351e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:20.605024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53a3ec-0c66-40f7-8704-35aeff6351e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:20.607007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53a3ec-0c66-40f7-8704-35aeff6351e2 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:20.709069 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1061/2135] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:20 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:20.713109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180b9a8-edbd-4726-a8d9-1d6b61974727 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:20.723837 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180b9a8-edbd-4726-a8d9-1d6b61974727 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:20.728499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7234db1-a9e2-4080-b069-2309c6361522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:20.729093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7234db1-a9e2-4080-b069-2309c6361522 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:20.729373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7234db1-a9e2-4080-b069-2309c6361522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:20.729647 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7234db1-a9e2-4080-b069-2309c6361522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:20.730317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7234db1-a9e2-4080-b069-2309c6361522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:20.731878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7234db1-a9e2-4080-b069-2309c6361522 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:20.741584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1075/2136] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:20.745560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53a3ec-0c66-40f7-8704-35aeff6351e2 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:20.755829 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53a3ec-0c66-40f7-8704-35aeff6351e2 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:20.760798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a613b-a371-4e0d-a436-c48b8f0f8e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:20.761407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a613b-a371-4e0d-a436-c48b8f0f8e0d admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:20.762187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a613b-a371-4e0d-a436-c48b8f0f8e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:20.762593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a613b-a371-4e0d-a436-c48b8f0f8e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:20.763250 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a613b-a371-4e0d-a436-c48b8f0f8e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:20.765188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a613b-a371-4e0d-a436-c48b8f0f8e0d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:20.780750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1062/2137] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:20.784371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7234db1-a9e2-4080-b069-2309c6361522 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:20.795097 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7234db1-a9e2-4080-b069-2309c6361522 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:20.800239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef46553-235a-407b-adae-abfb550591be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:20.800807 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:20.801085 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef46553-235a-407b-adae-abfb550591be admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:20.801439 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef46553-235a-407b-adae-abfb550591be admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:20.801782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef46553-235a-407b-adae-abfb550591be admin admin] PATH_INFO: `/v3/projects/89ed0194a8bf4b9fbba8557948fbd00e/users/64ec0b9aca8044738283cdbf5db17288/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:20.818121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef46553-235a-407b-adae-abfb550591be admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ed0194a8bf4b9fbba8557948fbd00e, user_id=64ec0b9aca8044738283cdbf5db17288, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:20.820951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef46553-235a-407b-adae-abfb550591be admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:20.921138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1076/2138] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:20 2022] PUT /identity/v3/projects/89ed0194a8bf4b9fbba8557948fbd00e/users/64ec0b9aca8044738283cdbf5db17288/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:20.932585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c7c88-2178-4c29-97e4-ee08c25b2559 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:20.932876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c7c88-2178-4c29-97e4-ee08c25b2559 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:20.933158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c7c88-2178-4c29-97e4-ee08c25b2559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:20.961642 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-0a9c7c88-2178-4c29-97e4-ee08c25b2559 None None] MFA Rules not processed for user `64ec0b9aca8044738283cdbf5db17288`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:20.962850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0a9c7c88-2178-4c29-97e4-ee08c25b2559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:21.026377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1063/2139] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:21.031182 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef46553-235a-407b-adae-abfb550591be admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:21.093532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef46553-235a-407b-adae-abfb550591be admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:21.098142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c54c5-c2d2-41b1-b97f-3d26148aaa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:21.098739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c54c5-c2d2-41b1-b97f-3d26148aaa07 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:21.099020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c54c5-c2d2-41b1-b97f-3d26148aaa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:21.099295 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c54c5-c2d2-41b1-b97f-3d26148aaa07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:21.099920 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c54c5-c2d2-41b1-b97f-3d26148aaa07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:21.101860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c54c5-c2d2-41b1-b97f-3d26148aaa07 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:21.196259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1077/2140] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:21 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:21.200965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c7c88-2178-4c29-97e4-ee08c25b2559 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:21.211935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c7c88-2178-4c29-97e4-ee08c25b2559 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:21.257852 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:21.271823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def70c2-2f93-4dd3-8150-7994b878a5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:21.272468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def70c2-2f93-4dd3-8150-7994b878a5bc admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:21.272742 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def70c2-2f93-4dd3-8150-7994b878a5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:21.273020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def70c2-2f93-4dd3-8150-7994b878a5bc admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:21.273660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def70c2-2f93-4dd3-8150-7994b878a5bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:21.275590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def70c2-2f93-4dd3-8150-7994b878a5bc admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:21.406900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1064/2141] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:21 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:21.410822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c54c5-c2d2-41b1-b97f-3d26148aaa07 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:21.420911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c54c5-c2d2-41b1-b97f-3d26148aaa07 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:21.425489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1b309-64e1-43a0-bcc9-87731f47e29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:21.426123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1b309-64e1-43a0-bcc9-87731f47e29e admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:21.426405 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1b309-64e1-43a0-bcc9-87731f47e29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:21.432114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1b309-64e1-43a0-bcc9-87731f47e29e admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:21.432114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1b309-64e1-43a0-bcc9-87731f47e29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:21.432114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1b309-64e1-43a0-bcc9-87731f47e29e admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:21.439832 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1078/2142] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:21.443895 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def70c2-2f93-4dd3-8150-7994b878a5bc admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:21.457794 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def70c2-2f93-4dd3-8150-7994b878a5bc admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:21.460470 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f302f8-8280-4395-b01b-87d476c3b87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:21.461127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f302f8-8280-4395-b01b-87d476c3b87c admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:21.461406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f302f8-8280-4395-b01b-87d476c3b87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:21.461682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f302f8-8280-4395-b01b-87d476c3b87c admin admin] PATH_INFO: `/v3/projects/0021219a52d64671861e74d498e33cab/users/160be1b4faba4211afc8d5a4eab1ed37/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:21.478927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f302f8-8280-4395-b01b-87d476c3b87c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0021219a52d64671861e74d498e33cab, user_id=160be1b4faba4211afc8d5a4eab1ed37, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:21.481757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f302f8-8280-4395-b01b-87d476c3b87c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:21.589538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1065/2143] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:21 2022] PUT /identity/v3/projects/0021219a52d64671861e74d498e33cab/users/160be1b4faba4211afc8d5a4eab1ed37/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:21.594076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1b309-64e1-43a0-bcc9-87731f47e29e admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:21.604584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1b309-64e1-43a0-bcc9-87731f47e29e admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:21.669921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcf7c3-b029-49af-8f4f-47d67b56c862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:21.671019 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcf7c3-b029-49af-8f4f-47d67b56c862 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:21.671298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcf7c3-b029-49af-8f4f-47d67b56c862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:21.671597 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcf7c3-b029-49af-8f4f-47d67b56c862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:21.672302 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efcf7c3-b029-49af-8f4f-47d67b56c862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:21.674222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efcf7c3-b029-49af-8f4f-47d67b56c862 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:21.685034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1079/2144] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:21.689452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f302f8-8280-4395-b01b-87d476c3b87c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:21.699700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f302f8-8280-4395-b01b-87d476c3b87c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:21.704497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365e576-a338-4843-837c-4bd717f32ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:21.705143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365e576-a338-4843-837c-4bd717f32ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:21.705428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365e576-a338-4843-837c-4bd717f32ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:21.705704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365e576-a338-4843-837c-4bd717f32ec2 admin admin] PATH_INFO: `/v3/projects/0021219a52d64671861e74d498e33cab/users/160be1b4faba4211afc8d5a4eab1ed37/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:21.707753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365e576-a338-4843-837c-4bd717f32ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0021219a52d64671861e74d498e33cab, user_id=160be1b4faba4211afc8d5a4eab1ed37, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:21.710512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365e576-a338-4843-837c-4bd717f32ec2 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:21.721264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1066/2145] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:21 2022] PUT /identity/v3/projects/0021219a52d64671861e74d498e33cab/users/160be1b4faba4211afc8d5a4eab1ed37/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:21.731774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c097c3-6a54-4155-be8d-9d5af1b89a6f None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:21.732057 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c097c3-6a54-4155-be8d-9d5af1b89a6f None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:21.732366 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c097c3-6a54-4155-be8d-9d5af1b89a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:21.762218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d4c097c3-6a54-4155-be8d-9d5af1b89a6f None None] MFA Rules not processed for user `160be1b4faba4211afc8d5a4eab1ed37`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:21.763751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d4c097c3-6a54-4155-be8d-9d5af1b89a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:21.838360 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1080/2146] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:21 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:21.849903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365e576-a338-4843-837c-4bd717f32ec2 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:21.851657 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b365e576-a338-4843-837c-4bd717f32ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:21.867610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:21.923144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365e576-a338-4843-837c-4bd717f32ec2 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:21.926817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8b27c-995a-47e9-b876-56ae3b0f11d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:21.927417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8b27c-995a-47e9-b876-56ae3b0f11d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:21.927700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8b27c-995a-47e9-b876-56ae3b0f11d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:21.927977 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8b27c-995a-47e9-b876-56ae3b0f11d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:21.936904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8b27c-995a-47e9-b876-56ae3b0f11d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:21.938328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8b27c-995a-47e9-b876-56ae3b0f11d0 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:21.947461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1067/2147] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:22.086033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c097c3-6a54-4155-be8d-9d5af1b89a6f None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:22.147282 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c097c3-6a54-4155-be8d-9d5af1b89a6f None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:22.151676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:22.153053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:22.153339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:22.153614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] PATH_INFO: `/v3/users/64ec0b9aca8044738283cdbf5db17288` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:22.154845 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ec0b9aca8044738283cdbf5db17288)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:22.156828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:22.375428 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:22.389795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ec0b9aca8044738283cdbf5db17288'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:22.403507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ec0b9aca8044738283cdbf5db17288'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:22.560435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ec0b9aca8044738283cdbf5db17288'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:22.561250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ec0b9aca8044738283cdbf5db17288'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:22.574299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ec0b9aca8044738283cdbf5db17288'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:22.581557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1081/2148] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:22 2022] DELETE /identity/v3/users/64ec0b9aca8044738283cdbf5db17288 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:22.587133 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2850f2-8929-4143-896d-7074c7cc0658 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:22.587564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2850f2-8929-4143-896d-7074c7cc0658 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:22.587716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2850f2-8929-4143-896d-7074c7cc0658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:22.610607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-8b2850f2-8929-4143-896d-7074c7cc0658 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:22.611716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8b2850f2-8929-4143-896d-7074c7cc0658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:22.676763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1068/2149] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:22.686970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:22.688521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:22.748220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa3fd4-84db-4894-833f-3b917ae2d8a2 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:22.752271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0cb90-e885-41fb-a44a-c32881019cee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:22.752901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0cb90-e885-41fb-a44a-c32881019cee service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:22.753186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0cb90-e885-41fb-a44a-c32881019cee service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:22.753461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0cb90-e885-41fb-a44a-c32881019cee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:22.763672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d0cb90-e885-41fb-a44a-c32881019cee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:22.765275 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d0cb90-e885-41fb-a44a-c32881019cee service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:22.777789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1082/2150] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:23.149271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:23.510723 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2850f2-8929-4143-896d-7074c7cc0658 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:23.512812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8b2850f2-8929-4143-896d-7074c7cc0658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:23.525673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2850f2-8929-4143-896d-7074c7cc0658 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:23.549620 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:23.550314 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:23.550593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:23.550868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] PATH_INFO: `/v3/projects/89ed0194a8bf4b9fbba8557948fbd00e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:23.552358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ed0194a8bf4b9fbba8557948fbd00e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:23.557110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:23.753585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ed0194a8bf4b9fbba8557948fbd00e', 'request_id': 'req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:23.770067 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ed0194a8bf4b9fbba8557948fbd00e', 'request_id': 'req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:23.799328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:23.908484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ed0194a8bf4b9fbba8557948fbd00e', 'request_id': 'req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:23.920649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] The token cache is being invalidate because project 89ed0194a8bf4b9fbba8557948fbd00e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:23.920946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:23.921587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:23.923061 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1069/2151] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:23 2022] DELETE /identity/v3/projects/89ed0194a8bf4b9fbba8557948fbd00e => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:23.927071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0cb90-e885-41fb-a44a-c32881019cee service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:23.937113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0cb90-e885-41fb-a44a-c32881019cee service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:23.993878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85f21c-eddf-4089-a758-19cd0d399c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:23.994416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85f21c-eddf-4089-a758-19cd0d399c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:23.994700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85f21c-eddf-4089-a758-19cd0d399c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:23.994976 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85f21c-eddf-4089-a758-19cd0d399c33 admin admin] PATH_INFO: `/v3/users/160be1b4faba4211afc8d5a4eab1ed37` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:23.996100 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85f21c-eddf-4089-a758-19cd0d399c33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=160be1b4faba4211afc8d5a4eab1ed37)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:23.998046 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85f21c-eddf-4089-a758-19cd0d399c33 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:24.364572 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cd85f21c-eddf-4089-a758-19cd0d399c33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '160be1b4faba4211afc8d5a4eab1ed37'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:24.373732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cd85f21c-eddf-4089-a758-19cd0d399c33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '160be1b4faba4211afc8d5a4eab1ed37'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:24.435405 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:24.469099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cd85f21c-eddf-4089-a758-19cd0d399c33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '160be1b4faba4211afc8d5a4eab1ed37'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:24.469719 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cd85f21c-eddf-4089-a758-19cd0d399c33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '160be1b4faba4211afc8d5a4eab1ed37'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:24.483131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cd85f21c-eddf-4089-a758-19cd0d399c33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '160be1b4faba4211afc8d5a4eab1ed37'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:24.489504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1083/2152] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:23 2022] DELETE /identity/v3/users/160be1b4faba4211afc8d5a4eab1ed37 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:25.162032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:25.163544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:25.181357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e853c-77f4-4ce4-a7e6-b3038902c4a1 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:25.266658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc8b40-ab77-49b1-8640-00e5f560d332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:25.267841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc8b40-ab77-49b1-8640-00e5f560d332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:25.268627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc8b40-ab77-49b1-8640-00e5f560d332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:25.269912 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc8b40-ab77-49b1-8640-00e5f560d332 admin admin] PATH_INFO: `/v3/projects/0021219a52d64671861e74d498e33cab` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:25.272141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc8b40-ab77-49b1-8640-00e5f560d332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0021219a52d64671861e74d498e33cab)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:25.274865 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc8b40-ab77-49b1-8640-00e5f560d332 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:25.425659 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c0dc8b40-ab77-49b1-8640-00e5f560d332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0021219a52d64671861e74d498e33cab', 'request_id': 'req-c0dc8b40-ab77-49b1-8640-00e5f560d332'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:25.436107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c0dc8b40-ab77-49b1-8640-00e5f560d332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0021219a52d64671861e74d498e33cab', 'request_id': 'req-c0dc8b40-ab77-49b1-8640-00e5f560d332'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:25.509504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c0dc8b40-ab77-49b1-8640-00e5f560d332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0021219a52d64671861e74d498e33cab', 'request_id': 'req-c0dc8b40-ab77-49b1-8640-00e5f560d332'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:25.525965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c0dc8b40-ab77-49b1-8640-00e5f560d332 admin admin] The token cache is being invalidate because project 0021219a52d64671861e74d498e33cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:25.526575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c0dc8b40-ab77-49b1-8640-00e5f560d332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:25.527292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c0dc8b40-ab77-49b1-8640-00e5f560d332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:25.528869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1070/2153] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:25 2022] DELETE /identity/v3/projects/0021219a52d64671861e74d498e33cab => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:25.533825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56dd4a-ea86-4287-b523-7bdaec63162c None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:25.534287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56dd4a-ea86-4287-b523-7bdaec63162c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:25.534518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56dd4a-ea86-4287-b523-7bdaec63162c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:25.557843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-6f56dd4a-ea86-4287-b523-7bdaec63162c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:25.560679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-6f56dd4a-ea86-4287-b523-7bdaec63162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:25.591229 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:25.641346 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1084/2154] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:25.649456 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d9fec-bbfb-4dc9-992b-7e65d6658952 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:25.649879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d9fec-bbfb-4dc9-992b-7e65d6658952 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:25.650318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d9fec-bbfb-4dc9-992b-7e65d6658952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:25.670914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-bb1d9fec-bbfb-4dc9-992b-7e65d6658952 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:25.672149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-bb1d9fec-bbfb-4dc9-992b-7e65d6658952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:25.680371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1071/2155] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:25.685338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56dd4a-ea86-4287-b523-7bdaec63162c None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:25.695557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56dd4a-ea86-4287-b523-7bdaec63162c None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:25.701145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed69e7a-9ec3-4a53-b42a-7a8f621da025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:25.701802 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed69e7a-9ec3-4a53-b42a-7a8f621da025 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:25.702055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed69e7a-9ec3-4a53-b42a-7a8f621da025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:25.702335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed69e7a-9ec3-4a53-b42a-7a8f621da025 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:25.704357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed69e7a-9ec3-4a53-b42a-7a8f621da025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:25.704357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed69e7a-9ec3-4a53-b42a-7a8f621da025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:25.705544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed69e7a-9ec3-4a53-b42a-7a8f621da025 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:25.719180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1085/2156] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:28:25.723726 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d9fec-bbfb-4dc9-992b-7e65d6658952 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:25.734341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d9fec-bbfb-4dc9-992b-7e65d6658952 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:25.742405 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e3b8c-6633-4b45-a39b-d05901c6dd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:25.743465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e3b8c-6633-4b45-a39b-d05901c6dd9d admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:25.743814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e3b8c-6633-4b45-a39b-d05901c6dd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:25.744192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e3b8c-6633-4b45-a39b-d05901c6dd9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:25.744860 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e3b8c-6633-4b45-a39b-d05901c6dd9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:25.747492 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e3b8c-6633-4b45-a39b-d05901c6dd9d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:25.817615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:25.875694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1072/2157] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:25 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:25.879959 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed69e7a-9ec3-4a53-b42a-7a8f621da025 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:25.892381 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed69e7a-9ec3-4a53-b42a-7a8f621da025 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:25.951426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b3055-ac27-48ab-9f30-6012d77260f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:25.952037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b3055-ac27-48ab-9f30-6012d77260f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:25.952344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b3055-ac27-48ab-9f30-6012d77260f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:25.952666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b3055-ac27-48ab-9f30-6012d77260f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:25.953372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b3055-ac27-48ab-9f30-6012d77260f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:25.955218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b3055-ac27-48ab-9f30-6012d77260f4 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:26.140301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1086/2158] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:25 2022] POST /identity/v3/users => generated 303 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:26.144670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e3b8c-6633-4b45-a39b-d05901c6dd9d admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:26.156141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e3b8c-6633-4b45-a39b-d05901c6dd9d admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:26.160833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4198a-5110-4350-a0a8-3c207ec7588b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:26.161432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4198a-5110-4350-a0a8-3c207ec7588b admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:26.161709 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4198a-5110-4350-a0a8-3c207ec7588b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:26.161978 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4198a-5110-4350-a0a8-3c207ec7588b admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:26.162623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4198a-5110-4350-a0a8-3c207ec7588b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:26.164576 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4198a-5110-4350-a0a8-3c207ec7588b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:26.179669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1073/2159] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:26.180310 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b3055-ac27-48ab-9f30-6012d77260f4 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:26.200611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b3055-ac27-48ab-9f30-6012d77260f4 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:26.205321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271a9fd-4035-40d2-9049-6aa94afb4694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:26.205925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271a9fd-4035-40d2-9049-6aa94afb4694 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:26.206204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271a9fd-4035-40d2-9049-6aa94afb4694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:26.206472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271a9fd-4035-40d2-9049-6aa94afb4694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:26.207120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271a9fd-4035-40d2-9049-6aa94afb4694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:26.208739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271a9fd-4035-40d2-9049-6aa94afb4694 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:26.223688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1087/2160] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:28:26.224930 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4198a-5110-4350-a0a8-3c207ec7588b admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:26.234786 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4198a-5110-4350-a0a8-3c207ec7588b admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:26.239577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a81e2d-d4c4-4407-ac9b-ac37562208f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:26.240272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a81e2d-d4c4-4407-ac9b-ac37562208f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:26.240571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a81e2d-d4c4-4407-ac9b-ac37562208f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:26.240840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a81e2d-d4c4-4407-ac9b-ac37562208f3 admin admin] PATH_INFO: `/v3/projects/b127f4344e32491b9d7d3bf4abb0f527/users/c060178cf4e5421bb794eee78bed9c9a/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:26.256122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a81e2d-d4c4-4407-ac9b-ac37562208f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b127f4344e32491b9d7d3bf4abb0f527, user_id=c060178cf4e5421bb794eee78bed9c9a, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:26.258932 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a81e2d-d4c4-4407-ac9b-ac37562208f3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:26.355145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1074/2161] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:26 2022] PUT /identity/v3/projects/b127f4344e32491b9d7d3bf4abb0f527/users/c060178cf4e5421bb794eee78bed9c9a/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:26.366821 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267112e-dd8e-4a78-98fb-6093678b5ca0 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:26.367139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267112e-dd8e-4a78-98fb-6093678b5ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:26.367446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267112e-dd8e-4a78-98fb-6093678b5ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:26.402472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-5267112e-dd8e-4a78-98fb-6093678b5ca0 None None] MFA Rules not processed for user `c060178cf4e5421bb794eee78bed9c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:26.402472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5267112e-dd8e-4a78-98fb-6093678b5ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:26.474399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1088/2162] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:26 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:26.478722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a81e2d-d4c4-4407-ac9b-ac37562208f3 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:26.591002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a81e2d-d4c4-4407-ac9b-ac37562208f3 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:26.596416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f1f43-ab3e-4433-96b3-4e406fc8a8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:26.597162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f1f43-ab3e-4433-96b3-4e406fc8a8a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:26.597577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f1f43-ab3e-4433-96b3-4e406fc8a8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:26.598005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f1f43-ab3e-4433-96b3-4e406fc8a8a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:26.599053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f1f43-ab3e-4433-96b3-4e406fc8a8a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:26.608670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f1f43-ab3e-4433-96b3-4e406fc8a8a3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:26.775234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267112e-dd8e-4a78-98fb-6093678b5ca0 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:26.786064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1075/2163] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:26 2022] POST /identity/v3/projects => generated 407 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:26.789993 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267112e-dd8e-4a78-98fb-6093678b5ca0 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:26.863004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e9852-249a-4c7d-8605-fcf69464f563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:26.863646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e9852-249a-4c7d-8605-fcf69464f563 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:26.863928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e9852-249a-4c7d-8605-fcf69464f563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:26.864231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e9852-249a-4c7d-8605-fcf69464f563 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:26.864885 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e9852-249a-4c7d-8605-fcf69464f563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:26.866877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e9852-249a-4c7d-8605-fcf69464f563 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:26.978214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1089/2164] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:26 2022] POST /identity/v3/users => generated 310 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:26.982925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f1f43-ab3e-4433-96b3-4e406fc8a8a3 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:26.994846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f1f43-ab3e-4433-96b3-4e406fc8a8a3 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:26.999944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47eefec-a483-4ae2-9b9c-366cc02da7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:27.002785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47eefec-a483-4ae2-9b9c-366cc02da7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:27.002785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47eefec-a483-4ae2-9b9c-366cc02da7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:27.002785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47eefec-a483-4ae2-9b9c-366cc02da7a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:27.002785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47eefec-a483-4ae2-9b9c-366cc02da7a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:27.003458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47eefec-a483-4ae2-9b9c-366cc02da7a7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:27.015825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1076/2165] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:27.020119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e9852-249a-4c7d-8605-fcf69464f563 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:27.032579 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e9852-249a-4c7d-8605-fcf69464f563 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:27.037511 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629859bc-f5a7-45be-ae32-ca8d859934ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:27.038170 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629859bc-f5a7-45be-ae32-ca8d859934ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:27.038449 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629859bc-f5a7-45be-ae32-ca8d859934ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:27.038733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629859bc-f5a7-45be-ae32-ca8d859934ec admin admin] PATH_INFO: `/v3/projects/d0fe989af30c4e15b96833e9773b6a94/users/50b02dcb74f14237acb66629a88d894e/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:27.059307 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629859bc-f5a7-45be-ae32-ca8d859934ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0fe989af30c4e15b96833e9773b6a94, user_id=50b02dcb74f14237acb66629a88d894e, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:27.062651 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629859bc-f5a7-45be-ae32-ca8d859934ec admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:27.101157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:27.169693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1090/2166] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:27 2022] PUT /identity/v3/projects/d0fe989af30c4e15b96833e9773b6a94/users/50b02dcb74f14237acb66629a88d894e/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:27.171295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47eefec-a483-4ae2-9b9c-366cc02da7a7 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:27.181140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47eefec-a483-4ae2-9b9c-366cc02da7a7 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:27.245325 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f1161-89ef-44f6-95d7-7a1d3086537c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:27.245963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f1161-89ef-44f6-95d7-7a1d3086537c admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:27.246416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f1161-89ef-44f6-95d7-7a1d3086537c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:27.246728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f1161-89ef-44f6-95d7-7a1d3086537c admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:27.247379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f1161-89ef-44f6-95d7-7a1d3086537c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:27.249015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f1161-89ef-44f6-95d7-7a1d3086537c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:27.257520 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:27.261071 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1077/2167] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:27.264454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629859bc-f5a7-45be-ae32-ca8d859934ec admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:27.274899 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629859bc-f5a7-45be-ae32-ca8d859934ec admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:27.280221 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d17b89-b0ef-4375-b6bc-209e35651cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:27.280849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d17b89-b0ef-4375-b6bc-209e35651cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:27.281129 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d17b89-b0ef-4375-b6bc-209e35651cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:27.281404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d17b89-b0ef-4375-b6bc-209e35651cc7 admin admin] PATH_INFO: `/v3/projects/d0fe989af30c4e15b96833e9773b6a94/users/50b02dcb74f14237acb66629a88d894e/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:27.283393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d17b89-b0ef-4375-b6bc-209e35651cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0fe989af30c4e15b96833e9773b6a94, user_id=50b02dcb74f14237acb66629a88d894e, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:27.286150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d17b89-b0ef-4375-b6bc-209e35651cc7 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:27.297093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1091/2168] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:27 2022] PUT /identity/v3/projects/d0fe989af30c4e15b96833e9773b6a94/users/50b02dcb74f14237acb66629a88d894e/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:27.307387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9474d17-d030-4286-8799-47e4d08d45fd None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:27.307674 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9474d17-d030-4286-8799-47e4d08d45fd None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:27.307977 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9474d17-d030-4286-8799-47e4d08d45fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:27.338413 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f9474d17-d030-4286-8799-47e4d08d45fd None None] MFA Rules not processed for user `50b02dcb74f14237acb66629a88d894e`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:27.341261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f9474d17-d030-4286-8799-47e4d08d45fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:27.352465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d17b89-b0ef-4375-b6bc-209e35651cc7 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:27.354762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e2d17b89-b0ef-4375-b6bc-209e35651cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:27.421680 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1078/2169] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:27 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:27.432285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9474d17-d030-4286-8799-47e4d08d45fd None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:27.434357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f9474d17-d030-4286-8799-47e4d08d45fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:27.434611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d17b89-b0ef-4375-b6bc-209e35651cc7 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:27.439549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:27.440188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:27.440784 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:27.440784 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] PATH_INFO: `/v3/users/c823ba2f1a1b441989d27846a43d60ad` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:27.442895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c823ba2f1a1b441989d27846a43d60ad)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:27.443814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:27.489217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9474d17-d030-4286-8799-47e4d08d45fd None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:27.492796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45512f6-f759-40a6-9d58-b486a937ce4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:27.493390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45512f6-f759-40a6-9d58-b486a937ce4e service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:27.493663 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45512f6-f759-40a6-9d58-b486a937ce4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:27.493928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45512f6-f759-40a6-9d58-b486a937ce4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:27.503532 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45512f6-f759-40a6-9d58-b486a937ce4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:27.505021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45512f6-f759-40a6-9d58-b486a937ce4e service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:27.514550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1079/2170] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:27 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:28:27.636424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c823ba2f1a1b441989d27846a43d60ad'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:27.643397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c823ba2f1a1b441989d27846a43d60ad'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:27.737547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c823ba2f1a1b441989d27846a43d60ad'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:27.738139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c823ba2f1a1b441989d27846a43d60ad'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:27.750154 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c823ba2f1a1b441989d27846a43d60ad'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:27.755995 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1092/2171] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:27 2022] DELETE /identity/v3/users/c823ba2f1a1b441989d27846a43d60ad => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:27.998939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45512f6-f759-40a6-9d58-b486a937ce4e service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:28.007386 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45512f6-f759-40a6-9d58-b486a937ce4e service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:28.058109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d7f7a-882e-47fd-adf4-1e65586b1b8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:28.058716 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d7f7a-882e-47fd-adf4-1e65586b1b8e service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:28.058997 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d7f7a-882e-47fd-adf4-1e65586b1b8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:28.059271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d7f7a-882e-47fd-adf4-1e65586b1b8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:28.061134 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-037d7f7a-882e-47fd-adf4-1e65586b1b8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:28.083875 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:28.086091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:28.138283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d7f7a-882e-47fd-adf4-1e65586b1b8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:28.139781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d7f7a-882e-47fd-adf4-1e65586b1b8e service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:28.149920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1080/2172] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:27 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:28.164873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3192c-11d0-43c2-a40e-5b4faef4ee34 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:28.170082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901b2c0-608c-4e2f-b819-3896dfc7d08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:28.170977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901b2c0-608c-4e2f-b819-3896dfc7d08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:28.171271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901b2c0-608c-4e2f-b819-3896dfc7d08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:28.171551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901b2c0-608c-4e2f-b819-3896dfc7d08d admin admin] PATH_INFO: `/v3/projects/0eaee9c0d7014c88b9b08f4f5e020a66` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:28.174140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901b2c0-608c-4e2f-b819-3896dfc7d08d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eaee9c0d7014c88b9b08f4f5e020a66)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:28.176199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901b2c0-608c-4e2f-b819-3896dfc7d08d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:28.366640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2901b2c0-608c-4e2f-b819-3896dfc7d08d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eaee9c0d7014c88b9b08f4f5e020a66', 'request_id': 'req-2901b2c0-608c-4e2f-b819-3896dfc7d08d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:28.373372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2901b2c0-608c-4e2f-b819-3896dfc7d08d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eaee9c0d7014c88b9b08f4f5e020a66', 'request_id': 'req-2901b2c0-608c-4e2f-b819-3896dfc7d08d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:28.378763 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2901b2c0-608c-4e2f-b819-3896dfc7d08d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eaee9c0d7014c88b9b08f4f5e020a66', 'request_id': 'req-2901b2c0-608c-4e2f-b819-3896dfc7d08d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:28.509331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2901b2c0-608c-4e2f-b819-3896dfc7d08d admin admin] The token cache is being invalidate because project 0eaee9c0d7014c88b9b08f4f5e020a66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:28.509616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2901b2c0-608c-4e2f-b819-3896dfc7d08d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:28.510402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2901b2c0-608c-4e2f-b819-3896dfc7d08d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:28.512050 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1093/2173] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:28 2022] DELETE /identity/v3/projects/0eaee9c0d7014c88b9b08f4f5e020a66 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:28.517129 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51732e25-c39b-4953-b976-8c9b01f413df None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:28.517608 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51732e25-c39b-4953-b976-8c9b01f413df None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:28.517703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51732e25-c39b-4953-b976-8c9b01f413df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:28.538213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-51732e25-c39b-4953-b976-8c9b01f413df None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:28.543498 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-51732e25-c39b-4953-b976-8c9b01f413df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:28.601898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1081/2174] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:28:28.610414 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dcffc-5886-447e-8dea-0b9dd991820c None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:28.610903 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dcffc-5886-447e-8dea-0b9dd991820c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:28.611189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dcffc-5886-447e-8dea-0b9dd991820c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:28.632255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-5d8dcffc-5886-447e-8dea-0b9dd991820c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:28.633465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5d8dcffc-5886-447e-8dea-0b9dd991820c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:28.641689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1094/2175] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:28.646549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51732e25-c39b-4953-b976-8c9b01f413df None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:28.656905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51732e25-c39b-4953-b976-8c9b01f413df None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:28.661651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe87da4-807b-454d-aa21-43862e60654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:28.662263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe87da4-807b-454d-aa21-43862e60654a admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:28.662534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe87da4-807b-454d-aa21-43862e60654a admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:28.662810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe87da4-807b-454d-aa21-43862e60654a admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:28.663476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe87da4-807b-454d-aa21-43862e60654a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:28.663762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe87da4-807b-454d-aa21-43862e60654a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:28.665387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe87da4-807b-454d-aa21-43862e60654a admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:28.677330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1082/2176] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:28:28.681521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8dcffc-5886-447e-8dea-0b9dd991820c None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:28.691408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8dcffc-5886-447e-8dea-0b9dd991820c None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:28.696054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060230d7-a7f3-4407-afb7-3709371de125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:28.696689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060230d7-a7f3-4407-afb7-3709371de125 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:28.696969 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060230d7-a7f3-4407-afb7-3709371de125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:28.697237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060230d7-a7f3-4407-afb7-3709371de125 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:28.697886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060230d7-a7f3-4407-afb7-3709371de125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:28.700540 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060230d7-a7f3-4407-afb7-3709371de125 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:28.807657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1095/2177] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:28 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:28.812400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe87da4-807b-454d-aa21-43862e60654a admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:28.822248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe87da4-807b-454d-aa21-43862e60654a admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:28.879819 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e79e8-0f35-46b2-b8a3-393ffbf1e3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:28.880403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e79e8-0f35-46b2-b8a3-393ffbf1e3ad admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:28.880682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e79e8-0f35-46b2-b8a3-393ffbf1e3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:28.880958 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e79e8-0f35-46b2-b8a3-393ffbf1e3ad admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:28.881637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e79e8-0f35-46b2-b8a3-393ffbf1e3ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:28.883580 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e79e8-0f35-46b2-b8a3-393ffbf1e3ad admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:28.903767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:29.018123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1083/2178] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:28 2022] POST /identity/v3/users => generated 295 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:29.022462 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060230d7-a7f3-4407-afb7-3709371de125 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:29.033598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060230d7-a7f3-4407-afb7-3709371de125 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:29.038305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a738a-f457-4dd9-beca-a835e0a999c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:29.038945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a738a-f457-4dd9-beca-a835e0a999c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:29.039230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a738a-f457-4dd9-beca-a835e0a999c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:29.039552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a738a-f457-4dd9-beca-a835e0a999c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:29.040229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a738a-f457-4dd9-beca-a835e0a999c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:29.041863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a738a-f457-4dd9-beca-a835e0a999c1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:29.055001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:29.058279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1096/2179] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:29.058394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e79e8-0f35-46b2-b8a3-393ffbf1e3ad admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:29.083998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e79e8-0f35-46b2-b8a3-393ffbf1e3ad admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:29.083998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dacc8-4071-4d82-aea6-312dea42a7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:29.083998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dacc8-4071-4d82-aea6-312dea42a7ed admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:29.083998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dacc8-4071-4d82-aea6-312dea42a7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:29.084363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dacc8-4071-4d82-aea6-312dea42a7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:29.084363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dacc8-4071-4d82-aea6-312dea42a7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:29.084363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dacc8-4071-4d82-aea6-312dea42a7ed admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:29.088233 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1084/2180] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:29.092942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a738a-f457-4dd9-beca-a835e0a999c1 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:29.104832 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a738a-f457-4dd9-beca-a835e0a999c1 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:29.118860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687eae0-c3b0-47a0-89c4-975f3220487d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:29.118860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687eae0-c3b0-47a0-89c4-975f3220487d admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:29.118860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687eae0-c3b0-47a0-89c4-975f3220487d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:29.118860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687eae0-c3b0-47a0-89c4-975f3220487d admin admin] PATH_INFO: `/v3/projects/e6f51ffa08494524b47ac220f1bd74de/users/859d36aa86bd403ca1d46b245efaef61/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:29.127869 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687eae0-c3b0-47a0-89c4-975f3220487d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6f51ffa08494524b47ac220f1bd74de, user_id=859d36aa86bd403ca1d46b245efaef61, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:29.130877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687eae0-c3b0-47a0-89c4-975f3220487d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:29.264952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1097/2181] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:29 2022] PUT /identity/v3/projects/e6f51ffa08494524b47ac220f1bd74de/users/859d36aa86bd403ca1d46b245efaef61/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:29.270347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa39e5-de05-47e9-965c-4d0cbedda597 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:29.270740 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa39e5-de05-47e9-965c-4d0cbedda597 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:29.271147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa39e5-de05-47e9-965c-4d0cbedda597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:29.295495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-c9fa39e5-de05-47e9-965c-4d0cbedda597 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:29.297436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c9fa39e5-de05-47e9-965c-4d0cbedda597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:29.315175 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687eae0-c3b0-47a0-89c4-975f3220487d admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:29.317161 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a687eae0-c3b0-47a0-89c4-975f3220487d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:29.370554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1085/2182] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:29.382058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa39e5-de05-47e9-965c-4d0cbedda597 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:29.383476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c9fa39e5-de05-47e9-965c-4d0cbedda597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:29.402606 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687eae0-c3b0-47a0-89c4-975f3220487d admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:29.408136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:29.409071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:29.409476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:29.409860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] PATH_INFO: `/v3/users/cd4c1cd9fdc546f486138259bd9c4e4e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:29.411178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd4c1cd9fdc546f486138259bd9c4e4e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:29.413331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:29.417952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:29.444195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa39e5-de05-47e9-965c-4d0cbedda597 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:29.447789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0016e14-3b27-4271-abed-c518c28d47ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:29.448426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0016e14-3b27-4271-abed-c518c28d47ce service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:29.448714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0016e14-3b27-4271-abed-c518c28d47ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:29.449632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0016e14-3b27-4271-abed-c518c28d47ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:29.458583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0016e14-3b27-4271-abed-c518c28d47ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:29.460053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0016e14-3b27-4271-abed-c518c28d47ce service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:29.469296 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1086/2183] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:28:29.609584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd4c1cd9fdc546f486138259bd9c4e4e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:29.620207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd4c1cd9fdc546f486138259bd9c4e4e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:29.623728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:29.738087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd4c1cd9fdc546f486138259bd9c4e4e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:29.738718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd4c1cd9fdc546f486138259bd9c4e4e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:29.756146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd4c1cd9fdc546f486138259bd9c4e4e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:29.758716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1098/2184] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:29 2022] DELETE /identity/v3/users/cd4c1cd9fdc546f486138259bd9c4e4e => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:29.763523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46a1d6-a5f5-49e7-b96f-954a0388b471 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:29.763815 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46a1d6-a5f5-49e7-b96f-954a0388b471 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:29.764095 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46a1d6-a5f5-49e7-b96f-954a0388b471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:29.785879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-9c46a1d6-a5f5-49e7-b96f-954a0388b471 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:29.787199 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9c46a1d6-a5f5-49e7-b96f-954a0388b471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:29.851491 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1087/2185] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:28:29.862107 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:29.863670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:29.922788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99c66a-7f18-473d-9940-f0be43d509aa admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:29.926572 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cd833-aa1a-4b1c-9418-72d7d98a3e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:29.927171 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cd833-aa1a-4b1c-9418-72d7d98a3e09 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:29.927446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cd833-aa1a-4b1c-9418-72d7d98a3e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:29.927712 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cd833-aa1a-4b1c-9418-72d7d98a3e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:29.937360 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1cd833-aa1a-4b1c-9418-72d7d98a3e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:29.938817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1cd833-aa1a-4b1c-9418-72d7d98a3e09 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:29.948082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1099/2186] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:30.035581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:30.040758 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46a1d6-a5f5-49e7-b96f-954a0388b471 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:30.042188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9c46a1d6-a5f5-49e7-b96f-954a0388b471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:30.054393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46a1d6-a5f5-49e7-b96f-954a0388b471 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:30.059598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:30.060929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:30.060929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:30.060929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] PATH_INFO: `/v3/users/b09be3e57c644c938aae47464c9832d7` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:30.062023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b09be3e57c644c938aae47464c9832d7)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:30.063962 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:30.351417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b09be3e57c644c938aae47464c9832d7'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:30.352093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b09be3e57c644c938aae47464c9832d7'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:30.360089 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b09be3e57c644c938aae47464c9832d7'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:30.428699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b09be3e57c644c938aae47464c9832d7'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:30.429256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b09be3e57c644c938aae47464c9832d7'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:30.447238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1088/2187] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:30 2022] DELETE /identity/v3/users/b09be3e57c644c938aae47464c9832d7 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:30.452306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681f1ba-b64b-4377-a172-93e368eb5caf None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:30.452776 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681f1ba-b64b-4377-a172-93e368eb5caf None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:30.452866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681f1ba-b64b-4377-a172-93e368eb5caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:30.472694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-5681f1ba-b64b-4377-a172-93e368eb5caf None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:30.473946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5681f1ba-b64b-4377-a172-93e368eb5caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:30.487300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:30.488726 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:30.513846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:30.546973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1100/2188] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:30.558642 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681f1ba-b64b-4377-a172-93e368eb5caf None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:30.560683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5681f1ba-b64b-4377-a172-93e368eb5caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:30.568801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b05422-9ff2-4d9f-9f94-f8aef1d6b034 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:30.572473 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:30.573389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:30.573766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:30.574795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] PATH_INFO: `/v3/projects/c0f6de985fec4a64bfb79f755e9d4856` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:30.576192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0f6de985fec4a64bfb79f755e9d4856)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:30.578373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:30.579565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:30.628548 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681f1ba-b64b-4377-a172-93e368eb5caf None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:30.633106 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f6fbc-c0f7-4a0f-9d55-58513e78b538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:30.633721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f6fbc-c0f7-4a0f-9d55-58513e78b538 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:30.634009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f6fbc-c0f7-4a0f-9d55-58513e78b538 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:30.634286 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f6fbc-c0f7-4a0f-9d55-58513e78b538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:30.648064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f6fbc-c0f7-4a0f-9d55-58513e78b538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:30.649561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f6fbc-c0f7-4a0f-9d55-58513e78b538 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:30.659770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1101/2189] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:30.710442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0f6de985fec4a64bfb79f755e9d4856', 'request_id': 'req-c5627559-8467-4086-946d-1dea5fa350a3'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:30.716742 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0f6de985fec4a64bfb79f755e9d4856', 'request_id': 'req-c5627559-8467-4086-946d-1dea5fa350a3'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:30.824457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0f6de985fec4a64bfb79f755e9d4856', 'request_id': 'req-c5627559-8467-4086-946d-1dea5fa350a3'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:30.836761 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] The token cache is being invalidate because project c0f6de985fec4a64bfb79f755e9d4856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:30.837050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:30.837943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:30.840302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1089/2190] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:30 2022] DELETE /identity/v3/projects/c0f6de985fec4a64bfb79f755e9d4856 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:30.844504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f6fbc-c0f7-4a0f-9d55-58513e78b538 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:30.869233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f6fbc-c0f7-4a0f-9d55-58513e78b538 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:31.028386 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e06c2-c074-411b-88fe-85a5619eab48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:31.029090 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e06c2-c074-411b-88fe-85a5619eab48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:31.029377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e06c2-c074-411b-88fe-85a5619eab48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:31.030274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e06c2-c074-411b-88fe-85a5619eab48 admin admin] PATH_INFO: `/v3/users/da4d398460344216ac9aed36cd3efbff` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:31.031577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e06c2-c074-411b-88fe-85a5619eab48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da4d398460344216ac9aed36cd3efbff)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:31.034134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e06c2-c074-411b-88fe-85a5619eab48 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:31.169267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:31.170739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:31.195098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5627559-8467-4086-946d-1dea5fa350a3 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:31.201654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:31.202822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:31.203261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:31.203725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2 admin admin] PATH_INFO: `/v3/projects/1cd24239272b44388ddb1304fb184380` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:31.205673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd24239272b44388ddb1304fb184380)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:31.208556 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:31.276929 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-3e1e06c2-c074-411b-88fe-85a5619eab48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da4d398460344216ac9aed36cd3efbff'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:31.284149 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-3e1e06c2-c074-411b-88fe-85a5619eab48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da4d398460344216ac9aed36cd3efbff'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:31.356589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-3e1e06c2-c074-411b-88fe-85a5619eab48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da4d398460344216ac9aed36cd3efbff'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:31.357370 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-3e1e06c2-c074-411b-88fe-85a5619eab48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da4d398460344216ac9aed36cd3efbff'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:31.389353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-3e1e06c2-c074-411b-88fe-85a5619eab48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da4d398460344216ac9aed36cd3efbff'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:31.396603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1102/2191] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:30 2022] DELETE /identity/v3/users/da4d398460344216ac9aed36cd3efbff => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:31.400768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd24239272b44388ddb1304fb184380', 'request_id': 'req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:31.408063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd24239272b44388ddb1304fb184380', 'request_id': 'req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:31.497870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd24239272b44388ddb1304fb184380', 'request_id': 'req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:31.512639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2 admin admin] The token cache is being invalidate because project 1cd24239272b44388ddb1304fb184380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:31.513027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:31.513967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1a157642-62d7-4c6b-80b4-ce31f4b9b3a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:31.515514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1090/2192] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:31 2022] DELETE /identity/v3/projects/1cd24239272b44388ddb1304fb184380 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:31.520741 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff960ffa-fd68-437a-a258-9e2332dcbaf4 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:31.521217 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff960ffa-fd68-437a-a258-9e2332dcbaf4 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:31.521335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff960ffa-fd68-437a-a258-9e2332dcbaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:31.541736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ff960ffa-fd68-437a-a258-9e2332dcbaf4 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:31.543504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ff960ffa-fd68-437a-a258-9e2332dcbaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:31.625468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1103/2193] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:31.634094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04d022-7bab-461e-83be-83389302af0c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:31.634555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04d022-7bab-461e-83be-83389302af0c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:31.635015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04d022-7bab-461e-83be-83389302af0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:31.672418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-bb04d022-7bab-461e-83be-83389302af0c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:31.674107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-bb04d022-7bab-461e-83be-83389302af0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:31.705365 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1091/2194] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:31.709731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff960ffa-fd68-437a-a258-9e2332dcbaf4 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:31.721007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff960ffa-fd68-437a-a258-9e2332dcbaf4 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:31.727323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed31894-6107-44a1-bb90-7f31126e8d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:31.728027 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed31894-6107-44a1-bb90-7f31126e8d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:31.736066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed31894-6107-44a1-bb90-7f31126e8d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:31.736066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed31894-6107-44a1-bb90-7f31126e8d19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:31.736066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed31894-6107-44a1-bb90-7f31126e8d19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:31.736066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed31894-6107-44a1-bb90-7f31126e8d19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:31.736066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed31894-6107-44a1-bb90-7f31126e8d19 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:31.747515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1104/2195] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:28:31.752217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04d022-7bab-461e-83be-83389302af0c None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:31.763272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04d022-7bab-461e-83be-83389302af0c None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:31.771325 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b864168-7988-44b8-a0a9-c88bafde900c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:31.772049 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b864168-7988-44b8-a0a9-c88bafde900c admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:31.772363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b864168-7988-44b8-a0a9-c88bafde900c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:31.772683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b864168-7988-44b8-a0a9-c88bafde900c admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:31.773394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b864168-7988-44b8-a0a9-c88bafde900c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:31.775651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b864168-7988-44b8-a0a9-c88bafde900c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:31.931799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed31894-6107-44a1-bb90-7f31126e8d19 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:31.936031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-6ed31894-6107-44a1-bb90-7f31126e8d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:31.964478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1092/2196] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:31 2022] POST /identity/v3/projects => generated 399 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:31.971939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b864168-7988-44b8-a0a9-c88bafde900c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:31.978564 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed31894-6107-44a1-bb90-7f31126e8d19 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:32.094515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b864168-7988-44b8-a0a9-c88bafde900c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:32.097905 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:32.099313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:32.099589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:32.099706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117863e-fba1-4852-97f4-8a68a54977f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:32.099902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7 admin admin] PATH_INFO: `/v3/projects/5c0c01ccd4c849788b82ec4d99899047` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:32.100259 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117863e-fba1-4852-97f4-8a68a54977f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:32.101029 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117863e-fba1-4852-97f4-8a68a54977f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:32.101156 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0c01ccd4c849788b82ec4d99899047)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:32.101699 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117863e-fba1-4852-97f4-8a68a54977f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:32.102506 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117863e-fba1-4852-97f4-8a68a54977f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:32.104470 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117863e-fba1-4852-97f4-8a68a54977f3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:32.110148 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:32.411014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1093/2197] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:31 2022] POST /identity/v3/users => generated 300 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:32.415009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117863e-fba1-4852-97f4-8a68a54977f3 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:32.418727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0c01ccd4c849788b82ec4d99899047', 'request_id': 'req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:32.426385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0c01ccd4c849788b82ec4d99899047', 'request_id': 'req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:32.426629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117863e-fba1-4852-97f4-8a68a54977f3 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:32.432325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0c01ccd4c849788b82ec4d99899047', 'request_id': 'req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:32.484685 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5aee57-4ae2-4d7d-a0ad-bfe0c7b6d096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:32.485304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aee57-4ae2-4d7d-a0ad-bfe0c7b6d096 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:32.485587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aee57-4ae2-4d7d-a0ad-bfe0c7b6d096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:32.485860 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aee57-4ae2-4d7d-a0ad-bfe0c7b6d096 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:32.486524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5aee57-4ae2-4d7d-a0ad-bfe0c7b6d096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:32.488104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5aee57-4ae2-4d7d-a0ad-bfe0c7b6d096 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:32.501919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1094/2198] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:28:32.504780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5aee57-4ae2-4d7d-a0ad-bfe0c7b6d096 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:32.517784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5aee57-4ae2-4d7d-a0ad-bfe0c7b6d096 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:32.527479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726bee8-73a5-4fc1-bac1-9193926d0df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:32.528490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726bee8-73a5-4fc1-bac1-9193926d0df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:32.532439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726bee8-73a5-4fc1-bac1-9193926d0df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:32.532439 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726bee8-73a5-4fc1-bac1-9193926d0df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:32.535655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7 admin admin] The token cache is being invalidate because project 5c0c01ccd4c849788b82ec4d99899047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:32.535655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:32.536258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726bee8-73a5-4fc1-bac1-9193926d0df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:32.536358 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d7ff7d79-21c9-4f4b-ab12-e778ecec47f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:32.536452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726bee8-73a5-4fc1-bac1-9193926d0df1 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:32.536562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1105/2199] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:31 2022] DELETE /identity/v3/projects/5c0c01ccd4c849788b82ec4d99899047 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:32.537035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba861005-d305-41fc-9379-5dde4f6947ca None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:32.537325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba861005-d305-41fc-9379-5dde4f6947ca None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:32.539227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba861005-d305-41fc-9379-5dde4f6947ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:32.545198 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1095/2200] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:28:32.548921 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726bee8-73a5-4fc1-bac1-9193926d0df1 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:32.550812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0726bee8-73a5-4fc1-bac1-9193926d0df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:32.565612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726bee8-73a5-4fc1-bac1-9193926d0df1 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:32.566793 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ba861005-d305-41fc-9379-5dde4f6947ca None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:32.568097 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ba861005-d305-41fc-9379-5dde4f6947ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:32.571969 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d49ee-e018-468d-92ff-2c6dfe5d05cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:32.572906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d49ee-e018-468d-92ff-2c6dfe5d05cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:32.573192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d49ee-e018-468d-92ff-2c6dfe5d05cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:32.573493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d49ee-e018-468d-92ff-2c6dfe5d05cb admin admin] PATH_INFO: `/v3/projects/66fd885b97fb46ce9b8a1fd002a2757b/users/8601cac79b394be9bf48df3ef1ae89a6/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:32.578927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1106/2201] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:32.586980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7e763-7a7a-492f-8026-b5a89841a0aa None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:32.587503 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7e763-7a7a-492f-8026-b5a89841a0aa None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:32.587773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7e763-7a7a-492f-8026-b5a89841a0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:32.592980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d49ee-e018-468d-92ff-2c6dfe5d05cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=66fd885b97fb46ce9b8a1fd002a2757b, user_id=8601cac79b394be9bf48df3ef1ae89a6, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:32.595707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d49ee-e018-468d-92ff-2c6dfe5d05cb admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:32.621574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-4fb7e763-7a7a-492f-8026-b5a89841a0aa None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:32.622793 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-4fb7e763-7a7a-492f-8026-b5a89841a0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:32.630688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1107/2202] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:28:32.634933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7e763-7a7a-492f-8026-b5a89841a0aa None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:32.644732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7e763-7a7a-492f-8026-b5a89841a0aa None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:32.649497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af4c6b-e2f5-4500-a786-51e978a7da30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:32.650111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af4c6b-e2f5-4500-a786-51e978a7da30 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:32.650380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af4c6b-e2f5-4500-a786-51e978a7da30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:32.650645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af4c6b-e2f5-4500-a786-51e978a7da30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:32.651280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af4c6b-e2f5-4500-a786-51e978a7da30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:32.651574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af4c6b-e2f5-4500-a786-51e978a7da30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:32.653244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af4c6b-e2f5-4500-a786-51e978a7da30 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:32.665223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1108/2203] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:28:32.670144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af4c6b-e2f5-4500-a786-51e978a7da30 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:32.673062 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1096/2204] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:32 2022] PUT /identity/v3/projects/66fd885b97fb46ce9b8a1fd002a2757b/users/8601cac79b394be9bf48df3ef1ae89a6/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:32.680547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af4c6b-e2f5-4500-a786-51e978a7da30 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:32.693875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e0bfd-7b96-45f7-81e0-09750adcd21b None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:32.695027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e0bfd-7b96-45f7-81e0-09750adcd21b None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:32.695963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e0bfd-7b96-45f7-81e0-09750adcd21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:32.716890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:32.760546 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-329e0bfd-7b96-45f7-81e0-09750adcd21b None None] MFA Rules not processed for user `8601cac79b394be9bf48df3ef1ae89a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:32.761810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-329e0bfd-7b96-45f7-81e0-09750adcd21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:32.778254 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:32.783398 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501d171-f84e-4986-b776-bf2b76eac537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:32.784024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501d171-f84e-4986-b776-bf2b76eac537 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:32.784334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501d171-f84e-4986-b776-bf2b76eac537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:32.786260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501d171-f84e-4986-b776-bf2b76eac537 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:32.786923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501d171-f84e-4986-b776-bf2b76eac537 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:32.788913 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501d171-f84e-4986-b776-bf2b76eac537 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:32.841600 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1097/2205] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:32 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:32.863383 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e0bfd-7b96-45f7-81e0-09750adcd21b None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:32.866108 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-329e0bfd-7b96-45f7-81e0-09750adcd21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:32.882557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1109/2206] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:32 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:32.886334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501d171-f84e-4986-b776-bf2b76eac537 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:32.955886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e0bfd-7b96-45f7-81e0-09750adcd21b None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:32.993472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501d171-f84e-4986-b776-bf2b76eac537 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:33.003827 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c39b7-53be-45fb-86d0-71fb7163eca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:33.004555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c39b7-53be-45fb-86d0-71fb7163eca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:33.004803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c39b7-53be-45fb-86d0-71fb7163eca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:33.005087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c39b7-53be-45fb-86d0-71fb7163eca2 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:33.006067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c39b7-53be-45fb-86d0-71fb7163eca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:33.008079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c39b7-53be-45fb-86d0-71fb7163eca2 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:33.041231 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9260a8-6526-4f19-b08f-fed3958a40b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:33.041863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9260a8-6526-4f19-b08f-fed3958a40b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:33.042457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9260a8-6526-4f19-b08f-fed3958a40b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:33.042457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9260a8-6526-4f19-b08f-fed3958a40b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:33.112494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9260a8-6526-4f19-b08f-fed3958a40b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:33.114453 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9260a8-6526-4f19-b08f-fed3958a40b6 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:33.125851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1098/2207] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:32 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:33.156240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1110/2208] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:32 2022] POST /identity/v3/users => generated 292 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:33.164835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9260a8-6526-4f19-b08f-fed3958a40b6 service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:33.178714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9260a8-6526-4f19-b08f-fed3958a40b6 service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:33.184365 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce584a6-e815-4888-acd4-df5d45b2ed46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:33.185160 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce584a6-e815-4888-acd4-df5d45b2ed46 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:33.185452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce584a6-e815-4888-acd4-df5d45b2ed46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:33.185736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce584a6-e815-4888-acd4-df5d45b2ed46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:33.186376 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce584a6-e815-4888-acd4-df5d45b2ed46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:33.189443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce584a6-e815-4888-acd4-df5d45b2ed46 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:33.212087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1099/2209] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:28:33.216689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c39b7-53be-45fb-86d0-71fb7163eca2 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:33.230222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c39b7-53be-45fb-86d0-71fb7163eca2 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:33.234908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf022a3-1bb9-4749-abe5-6172f830194c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:33.235591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf022a3-1bb9-4749-abe5-6172f830194c admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:33.235938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf022a3-1bb9-4749-abe5-6172f830194c admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:33.236315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf022a3-1bb9-4749-abe5-6172f830194c admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:33.237033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf022a3-1bb9-4749-abe5-6172f830194c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:33.239940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf022a3-1bb9-4749-abe5-6172f830194c admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:33.265077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1111/2210] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:33.268080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce584a6-e815-4888-acd4-df5d45b2ed46 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:33.296430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce584a6-e815-4888-acd4-df5d45b2ed46 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:33.313565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29be607-c79f-4bc2-8840-cfb4f62b2492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:33.313565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29be607-c79f-4bc2-8840-cfb4f62b2492 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:33.313565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29be607-c79f-4bc2-8840-cfb4f62b2492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:33.313565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29be607-c79f-4bc2-8840-cfb4f62b2492 admin admin] PATH_INFO: `/v3/projects/eb737a2b0ca44ce6a6da2c40fba0d01b/users/ca9ffa72f7234863bc6f86943e8f978f/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:33.338791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29be607-c79f-4bc2-8840-cfb4f62b2492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb737a2b0ca44ce6a6da2c40fba0d01b, user_id=ca9ffa72f7234863bc6f86943e8f978f, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:33.343063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29be607-c79f-4bc2-8840-cfb4f62b2492 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:33.456531 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1100/2211] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:33 2022] PUT /identity/v3/projects/eb737a2b0ca44ce6a6da2c40fba0d01b/users/ca9ffa72f7234863bc6f86943e8f978f/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:33.459495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987f6c4-47ee-4eba-ae15-d12213d05283 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:33.459982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987f6c4-47ee-4eba-ae15-d12213d05283 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:33.460475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987f6c4-47ee-4eba-ae15-d12213d05283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:33.497910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-a987f6c4-47ee-4eba-ae15-d12213d05283 None None] MFA Rules not processed for user `ca9ffa72f7234863bc6f86943e8f978f`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:33.499236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a987f6c4-47ee-4eba-ae15-d12213d05283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:33.570894 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1112/2212] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:28:33.575329 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29be607-c79f-4bc2-8840-cfb4f62b2492 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:33.665064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29be607-c79f-4bc2-8840-cfb4f62b2492 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:33.678385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacef14e-0bae-49c1-9412-fd079cdfe5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:33.678385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacef14e-0bae-49c1-9412-fd079cdfe5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:33.678385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacef14e-0bae-49c1-9412-fd079cdfe5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:33.678385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacef14e-0bae-49c1-9412-fd079cdfe5a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:33.679169 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacef14e-0bae-49c1-9412-fd079cdfe5a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:33.681403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacef14e-0bae-49c1-9412-fd079cdfe5a7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:33.768055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1101/2213] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:33 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:33.772344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987f6c4-47ee-4eba-ae15-d12213d05283 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:33.783661 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987f6c4-47ee-4eba-ae15-d12213d05283 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:33.859490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c18ae-ff88-40d3-a245-dabf415c462f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:33.860211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c18ae-ff88-40d3-a245-dabf415c462f admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:33.860572 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c18ae-ff88-40d3-a245-dabf415c462f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:33.860913 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c18ae-ff88-40d3-a245-dabf415c462f admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:33.861681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c18ae-ff88-40d3-a245-dabf415c462f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:33.863677 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c18ae-ff88-40d3-a245-dabf415c462f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:33.978703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1113/2214] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:33 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:33.982649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacef14e-0bae-49c1-9412-fd079cdfe5a7 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:33.997928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacef14e-0bae-49c1-9412-fd079cdfe5a7 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:34.001088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5693670-5479-427a-a685-669c21878a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:34.001835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5693670-5479-427a-a685-669c21878a57 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:34.002229 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5693670-5479-427a-a685-669c21878a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:34.002647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5693670-5479-427a-a685-669c21878a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:34.003416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5693670-5479-427a-a685-669c21878a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:34.005443 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5693670-5479-427a-a685-669c21878a57 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:34.021308 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1102/2215] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:34.022353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c18ae-ff88-40d3-a245-dabf415c462f admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:34.033378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c18ae-ff88-40d3-a245-dabf415c462f admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:34.039318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6202e-cfa8-4a8c-b476-2f959a3e146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:34.039701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6202e-cfa8-4a8c-b476-2f959a3e146e admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:34.040053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6202e-cfa8-4a8c-b476-2f959a3e146e admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:34.040427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6202e-cfa8-4a8c-b476-2f959a3e146e admin admin] PATH_INFO: `/v3/projects/9a0603d989974a9083000c93ac8a4a67/users/b388fc29ef0945d7aedef993bc9df9fc/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:34.056475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6202e-cfa8-4a8c-b476-2f959a3e146e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0603d989974a9083000c93ac8a4a67, user_id=b388fc29ef0945d7aedef993bc9df9fc, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:34.059788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6202e-cfa8-4a8c-b476-2f959a3e146e admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:34.151050 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1114/2216] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:34 2022] PUT /identity/v3/projects/9a0603d989974a9083000c93ac8a4a67/users/b388fc29ef0945d7aedef993bc9df9fc/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:34.155150 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5693670-5479-427a-a685-669c21878a57 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:34.166067 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5693670-5479-427a-a685-669c21878a57 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:34.230862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd6e53-8125-4ab4-ad7b-0d529a6e2466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:34.231701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd6e53-8125-4ab4-ad7b-0d529a6e2466 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:34.231891 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd6e53-8125-4ab4-ad7b-0d529a6e2466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:34.232201 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd6e53-8125-4ab4-ad7b-0d529a6e2466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:34.232867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd6e53-8125-4ab4-ad7b-0d529a6e2466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:34.234510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd6e53-8125-4ab4-ad7b-0d529a6e2466 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:34.253248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1103/2217] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:34.262223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6202e-cfa8-4a8c-b476-2f959a3e146e admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:34.274898 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6202e-cfa8-4a8c-b476-2f959a3e146e admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:34.280446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b2eb9-aff0-4fc1-a28f-f80a4047c37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:34.281124 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b2eb9-aff0-4fc1-a28f-f80a4047c37c admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:34.281454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b2eb9-aff0-4fc1-a28f-f80a4047c37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:34.281733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b2eb9-aff0-4fc1-a28f-f80a4047c37c admin admin] PATH_INFO: `/v3/projects/9a0603d989974a9083000c93ac8a4a67/users/b388fc29ef0945d7aedef993bc9df9fc/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:34.283850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b2eb9-aff0-4fc1-a28f-f80a4047c37c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0603d989974a9083000c93ac8a4a67, user_id=b388fc29ef0945d7aedef993bc9df9fc, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:34.287585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b2eb9-aff0-4fc1-a28f-f80a4047c37c admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:34.311498 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1115/2218] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:34 2022] PUT /identity/v3/projects/9a0603d989974a9083000c93ac8a4a67/users/b388fc29ef0945d7aedef993bc9df9fc/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:34.322899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca1044-7a7c-490e-a569-71a646c72b8a None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:34.323192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca1044-7a7c-490e-a569-71a646c72b8a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:34.323478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca1044-7a7c-490e-a569-71a646c72b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:34.359689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-06ca1044-7a7c-490e-a569-71a646c72b8a None None] MFA Rules not processed for user `b388fc29ef0945d7aedef993bc9df9fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:34.361082 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-06ca1044-7a7c-490e-a569-71a646c72b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:34.455136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1104/2219] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:34.467355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b2eb9-aff0-4fc1-a28f-f80a4047c37c admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:34.553665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b2eb9-aff0-4fc1-a28f-f80a4047c37c admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:34.563728 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2843127-f334-4328-8e2f-e1d16dcb927f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:34.565818 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2843127-f334-4328-8e2f-e1d16dcb927f service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:34.569241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2843127-f334-4328-8e2f-e1d16dcb927f service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:34.569241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2843127-f334-4328-8e2f-e1d16dcb927f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:34.578204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2843127-f334-4328-8e2f-e1d16dcb927f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:34.586263 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2843127-f334-4328-8e2f-e1d16dcb927f service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:34.598870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1116/2220] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:34.772710 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:34.842692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca1044-7a7c-490e-a569-71a646c72b8a None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:34.842692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-06ca1044-7a7c-490e-a569-71a646c72b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:34.876772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca1044-7a7c-490e-a569-71a646c72b8a None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:34.885487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dba4dd-6e4c-4159-8b4f-9296895f2d49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:34.885487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dba4dd-6e4c-4159-8b4f-9296895f2d49 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:34.885487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dba4dd-6e4c-4159-8b4f-9296895f2d49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:34.885487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dba4dd-6e4c-4159-8b4f-9296895f2d49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:34.910715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:34.961962 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dba4dd-6e4c-4159-8b4f-9296895f2d49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:34.965113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dba4dd-6e4c-4159-8b4f-9296895f2d49 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:34.986223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1105/2221] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:35.600355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2843127-f334-4328-8e2f-e1d16dcb927f service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:35.689027 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2843127-f334-4328-8e2f-e1d16dcb927f service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:35.695215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:35.696598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:35.696598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:35.696598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] PATH_INFO: `/v3/users/ca9ffa72f7234863bc6f86943e8f978f` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:35.698366 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9ffa72f7234863bc6f86943e8f978f)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:35.700397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:35.998687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9ffa72f7234863bc6f86943e8f978f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:36.018097 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9ffa72f7234863bc6f86943e8f978f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:36.086746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9ffa72f7234863bc6f86943e8f978f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:36.087746 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9ffa72f7234863bc6f86943e8f978f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:36.107159 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9ffa72f7234863bc6f86943e8f978f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:36.113770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1117/2222] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:35 2022] DELETE /identity/v3/users/ca9ffa72f7234863bc6f86943e8f978f => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:36.124422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8749080-2dc9-45bf-b729-ff16f3df16be None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:36.126554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8749080-2dc9-45bf-b729-ff16f3df16be None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:36.126554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8749080-2dc9-45bf-b729-ff16f3df16be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:36.149404 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-a8749080-2dc9-45bf-b729-ff16f3df16be None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:36.150749 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a8749080-2dc9-45bf-b729-ff16f3df16be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:36.223511 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1106/2223] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:36.234158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:36.238847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:36.280429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ea13c-f3b7-4436-a34f-556bb96f95bd None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:36.280624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ea13c-f3b7-4436-a34f-556bb96f95bd None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:36.280879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ea13c-f3b7-4436-a34f-556bb96f95bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:36.302361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b2547-ba25-40cf-ae00-6cd7f7fd0eb5 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:36.306626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ff0a8-da81-4615-8fbe-6ac5836d4168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:36.307446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ff0a8-da81-4615-8fbe-6ac5836d4168 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:36.307734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ff0a8-da81-4615-8fbe-6ac5836d4168 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:36.308012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ff0a8-da81-4615-8fbe-6ac5836d4168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:36.317903 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ff0a8-da81-4615-8fbe-6ac5836d4168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:36.320515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ff0a8-da81-4615-8fbe-6ac5836d4168 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:36.321752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-477ea13c-f3b7-4436-a34f-556bb96f95bd None None] MFA Rules not processed for user `859d36aa86bd403ca1d46b245efaef61`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:36.323141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-477ea13c-f3b7-4436-a34f-556bb96f95bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:36.332476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1118/2224] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:28:36.412028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1107/2225] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:36.432440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ff0a8-da81-4615-8fbe-6ac5836d4168 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:36.432440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-462ff0a8-da81-4615-8fbe-6ac5836d4168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:36.505094 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ff0a8-da81-4615-8fbe-6ac5836d4168 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:36.510910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f677e3a-f888-4eb5-a40d-6b7b395e3b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:36.511632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f677e3a-f888-4eb5-a40d-6b7b395e3b11 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:36.511910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f677e3a-f888-4eb5-a40d-6b7b395e3b11 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:36.512210 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f677e3a-f888-4eb5-a40d-6b7b395e3b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:36.520699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f677e3a-f888-4eb5-a40d-6b7b395e3b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:36.522352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f677e3a-f888-4eb5-a40d-6b7b395e3b11 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:36.531724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1119/2226] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:36.692450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:36.739714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ea13c-f3b7-4436-a34f-556bb96f95bd None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:36.742207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-477ea13c-f3b7-4436-a34f-556bb96f95bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:36.780498 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ea13c-f3b7-4436-a34f-556bb96f95bd None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:36.786136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:36.786929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:36.787334 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:36.788221 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] PATH_INFO: `/v3/projects/eb737a2b0ca44ce6a6da2c40fba0d01b` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:36.789283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb737a2b0ca44ce6a6da2c40fba0d01b)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:36.791278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:36.834865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:36.920273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb737a2b0ca44ce6a6da2c40fba0d01b', 'request_id': 'req-4267ece8-4f22-400a-acc9-b0b8bdd18221'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:36.930148 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb737a2b0ca44ce6a6da2c40fba0d01b', 'request_id': 'req-4267ece8-4f22-400a-acc9-b0b8bdd18221'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:37.034438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb737a2b0ca44ce6a6da2c40fba0d01b', 'request_id': 'req-4267ece8-4f22-400a-acc9-b0b8bdd18221'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:37.050207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] The token cache is being invalidate because project eb737a2b0ca44ce6a6da2c40fba0d01b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:37.052343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:37.053078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:37.059368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f677e3a-f888-4eb5-a40d-6b7b395e3b11 service nova] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:37.059884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1108/2227] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:36 2022] DELETE /identity/v3/projects/eb737a2b0ca44ce6a6da2c40fba0d01b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:37.075190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f677e3a-f888-4eb5-a40d-6b7b395e3b11 service nova] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:37.149818 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbe858-71d1-42fc-9e5e-4e8b273e7282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:37.150812 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbe858-71d1-42fc-9e5e-4e8b273e7282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:37.151102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbe858-71d1-42fc-9e5e-4e8b273e7282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:37.151377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbe858-71d1-42fc-9e5e-4e8b273e7282 admin admin] PATH_INFO: `/v3/users/b388fc29ef0945d7aedef993bc9df9fc` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:37.152936 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbe858-71d1-42fc-9e5e-4e8b273e7282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b388fc29ef0945d7aedef993bc9df9fc)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:37.155566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbe858-71d1-42fc-9e5e-4e8b273e7282 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:37.442798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2bfbe858-71d1-42fc-9e5e-4e8b273e7282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b388fc29ef0945d7aedef993bc9df9fc'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:37.452145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2bfbe858-71d1-42fc-9e5e-4e8b273e7282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b388fc29ef0945d7aedef993bc9df9fc'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:37.520650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2bfbe858-71d1-42fc-9e5e-4e8b273e7282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b388fc29ef0945d7aedef993bc9df9fc'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:37.521416 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2bfbe858-71d1-42fc-9e5e-4e8b273e7282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b388fc29ef0945d7aedef993bc9df9fc'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:37.535250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2bfbe858-71d1-42fc-9e5e-4e8b273e7282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b388fc29ef0945d7aedef993bc9df9fc'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:37.542899 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1120/2228] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:37 2022] DELETE /identity/v3/users/b388fc29ef0945d7aedef993bc9df9fc => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:38.094871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:38.096411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:38.111262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267ece8-4f22-400a-acc9-b0b8bdd18221 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:38.176712 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b7851-7336-48ab-b1e1-763add12d69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:38.177406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b7851-7336-48ab-b1e1-763add12d69b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:38.177694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b7851-7336-48ab-b1e1-763add12d69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:38.177969 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b7851-7336-48ab-b1e1-763add12d69b admin admin] PATH_INFO: `/v3/projects/9a0603d989974a9083000c93ac8a4a67` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:38.179259 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b7851-7336-48ab-b1e1-763add12d69b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a0603d989974a9083000c93ac8a4a67)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:38.181251 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b7851-7336-48ab-b1e1-763add12d69b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:38.385896 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b63b7851-7336-48ab-b1e1-763add12d69b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a0603d989974a9083000c93ac8a4a67', 'request_id': 'req-b63b7851-7336-48ab-b1e1-763add12d69b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:38.408700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b63b7851-7336-48ab-b1e1-763add12d69b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a0603d989974a9083000c93ac8a4a67', 'request_id': 'req-b63b7851-7336-48ab-b1e1-763add12d69b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:38.491654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b63b7851-7336-48ab-b1e1-763add12d69b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a0603d989974a9083000c93ac8a4a67', 'request_id': 'req-b63b7851-7336-48ab-b1e1-763add12d69b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:38.517780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b63b7851-7336-48ab-b1e1-763add12d69b admin admin] The token cache is being invalidate because project 9a0603d989974a9083000c93ac8a4a67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:38.518593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b63b7851-7336-48ab-b1e1-763add12d69b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:38.520275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b63b7851-7336-48ab-b1e1-763add12d69b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:38.523785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1109/2229] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:38 2022] DELETE /identity/v3/projects/9a0603d989974a9083000c93ac8a4a67 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:38.529131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc3a33-5775-4e4e-99fd-f0fef70480f7 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:38.530109 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc3a33-5775-4e4e-99fd-f0fef70480f7 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:38.531063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc3a33-5775-4e4e-99fd-f0fef70480f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:38.573338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-f4dc3a33-5775-4e4e-99fd-f0fef70480f7 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:38.576004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-f4dc3a33-5775-4e4e-99fd-f0fef70480f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:38.672145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1121/2230] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:38.680307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872940b-aad5-45c4-be73-fbf0bfb0aa40 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:38.681030 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872940b-aad5-45c4-be73-fbf0bfb0aa40 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:38.681496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872940b-aad5-45c4-be73-fbf0bfb0aa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:38.714485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-6872940b-aad5-45c4-be73-fbf0bfb0aa40 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:38.715819 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-6872940b-aad5-45c4-be73-fbf0bfb0aa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:38.736593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1110/2231] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:38.739425 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc3a33-5775-4e4e-99fd-f0fef70480f7 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:38.750931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc3a33-5775-4e4e-99fd-f0fef70480f7 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:38.755577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0eb5f-9c11-4758-b2f1-77bdd5314a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:38.756337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0eb5f-9c11-4758-b2f1-77bdd5314a27 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:38.756626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0eb5f-9c11-4758-b2f1-77bdd5314a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:38.756905 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0eb5f-9c11-4758-b2f1-77bdd5314a27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:38.757561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0eb5f-9c11-4758-b2f1-77bdd5314a27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:38.757868 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0eb5f-9c11-4758-b2f1-77bdd5314a27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:38.759620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0eb5f-9c11-4758-b2f1-77bdd5314a27 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:38.772077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1122/2232] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:28:38.776847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872940b-aad5-45c4-be73-fbf0bfb0aa40 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:38.786387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872940b-aad5-45c4-be73-fbf0bfb0aa40 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:38.817537 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ade293-c169-4e74-8506-2d0daf55a480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:38.819774 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ade293-c169-4e74-8506-2d0daf55a480 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:38.820738 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ade293-c169-4e74-8506-2d0daf55a480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:38.822189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ade293-c169-4e74-8506-2d0daf55a480 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:38.824641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ade293-c169-4e74-8506-2d0daf55a480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:38.831317 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ade293-c169-4e74-8506-2d0daf55a480 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:39.022913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1111/2233] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:38 2022] POST /identity/v3/projects => generated 369 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:39.023095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0eb5f-9c11-4758-b2f1-77bdd5314a27 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:39.040032 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0eb5f-9c11-4758-b2f1-77bdd5314a27 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:39.120054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae20caa-529a-4d2b-a4dd-edd1dd93cea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:39.120772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae20caa-529a-4d2b-a4dd-edd1dd93cea1 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:39.121059 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae20caa-529a-4d2b-a4dd-edd1dd93cea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:39.121337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae20caa-529a-4d2b-a4dd-edd1dd93cea1 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:39.121997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae20caa-529a-4d2b-a4dd-edd1dd93cea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:39.124156 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae20caa-529a-4d2b-a4dd-edd1dd93cea1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:39.256832 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1123/2234] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:39 2022] POST /identity/v3/users => generated 285 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:39.260818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ade293-c169-4e74-8506-2d0daf55a480 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:39.284522 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ade293-c169-4e74-8506-2d0daf55a480 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:39.288003 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:39.299356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161151c-338b-4d05-858e-b790a3415890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:39.300283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161151c-338b-4d05-858e-b790a3415890 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:39.300737 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161151c-338b-4d05-858e-b790a3415890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:39.301184 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161151c-338b-4d05-858e-b790a3415890 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:39.302011 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161151c-338b-4d05-858e-b790a3415890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:39.304387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161151c-338b-4d05-858e-b790a3415890 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:39.316239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1112/2235] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:39.320020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae20caa-529a-4d2b-a4dd-edd1dd93cea1 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:39.332805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae20caa-529a-4d2b-a4dd-edd1dd93cea1 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:39.338788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a7df0-ab7c-43fe-8075-97288fb3862c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:39.340110 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a7df0-ab7c-43fe-8075-97288fb3862c admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:39.340513 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a7df0-ab7c-43fe-8075-97288fb3862c admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:39.340787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a7df0-ab7c-43fe-8075-97288fb3862c admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:39.341447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a7df0-ab7c-43fe-8075-97288fb3862c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:39.343292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a7df0-ab7c-43fe-8075-97288fb3862c admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:39.359025 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1124/2236] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:28:39.360652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161151c-338b-4d05-858e-b790a3415890 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:39.371833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161151c-338b-4d05-858e-b790a3415890 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:39.377271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b03214-f3a8-4876-a113-c99161a11c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:39.377935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b03214-f3a8-4876-a113-c99161a11c9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:39.378358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b03214-f3a8-4876-a113-c99161a11c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:39.378636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b03214-f3a8-4876-a113-c99161a11c9a admin admin] PATH_INFO: `/v3/projects/99524f37a64d4ab68595c236ff02cc72/users/97c047135ea541d892e0a02f00fa9bd8/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:39.396089 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b03214-f3a8-4876-a113-c99161a11c9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=99524f37a64d4ab68595c236ff02cc72, user_id=97c047135ea541d892e0a02f00fa9bd8, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:39.398982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b03214-f3a8-4876-a113-c99161a11c9a admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:39.530309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1113/2237] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:39 2022] PUT /identity/v3/projects/99524f37a64d4ab68595c236ff02cc72/users/97c047135ea541d892e0a02f00fa9bd8/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:39.532751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23bd5-a28f-46df-b28e-1c0afd8a33fe None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:39.533230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23bd5-a28f-46df-b28e-1c0afd8a33fe None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:39.533468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23bd5-a28f-46df-b28e-1c0afd8a33fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:39.566065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-cea23bd5-a28f-46df-b28e-1c0afd8a33fe None None] MFA Rules not processed for user `97c047135ea541d892e0a02f00fa9bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:39.567357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cea23bd5-a28f-46df-b28e-1c0afd8a33fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:39.631662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:39.664968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1125/2238] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:39.676992 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b03214-f3a8-4876-a113-c99161a11c9a admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:39.678764 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-70b03214-f3a8-4876-a113-c99161a11c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:39.713621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d6b8d-989b-4fd4-8a39-e22eb55a8c1a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:39.713909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d6b8d-989b-4fd4-8a39-e22eb55a8c1a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:39.714459 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d6b8d-989b-4fd4-8a39-e22eb55a8c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:39.750824 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b03214-f3a8-4876-a113-c99161a11c9a admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:39.755397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc029d1-c80d-438e-b226-1741d9d05b2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:39.756535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc029d1-c80d-438e-b226-1741d9d05b2b service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:39.757124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc029d1-c80d-438e-b226-1741d9d05b2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:39.757278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ee1d6b8d-989b-4fd4-8a39-e22eb55a8c1a None None] MFA Rules not processed for user `859d36aa86bd403ca1d46b245efaef61`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:39.757799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc029d1-c80d-438e-b226-1741d9d05b2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:39.758579 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ee1d6b8d-989b-4fd4-8a39-e22eb55a8c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:39.770086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc029d1-c80d-438e-b226-1741d9d05b2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:39.772878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc029d1-c80d-438e-b226-1741d9d05b2b service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:39.795501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1114/2239] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:39.819513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc029d1-c80d-438e-b226-1741d9d05b2b service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:39.855737 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1126/2240] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:28:39.914604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc029d1-c80d-438e-b226-1741d9d05b2b service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:39.919957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:39.920702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:39.920983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:39.921258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] PATH_INFO: `/v3/users/97c047135ea541d892e0a02f00fa9bd8` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:39.922430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c047135ea541d892e0a02f00fa9bd8)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:39.926175 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:40.178302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c047135ea541d892e0a02f00fa9bd8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:40.180225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c047135ea541d892e0a02f00fa9bd8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:40.198746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c047135ea541d892e0a02f00fa9bd8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:40.291381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c047135ea541d892e0a02f00fa9bd8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:40.292015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c047135ea541d892e0a02f00fa9bd8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:40.320285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1115/2241] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:39 2022] DELETE /identity/v3/users/97c047135ea541d892e0a02f00fa9bd8 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:40.325350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87cf3e-ca09-4121-8d1e-656bebceda7e None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:40.325655 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87cf3e-ca09-4121-8d1e-656bebceda7e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:40.325924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87cf3e-ca09-4121-8d1e-656bebceda7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:40.347390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-9a87cf3e-ca09-4121-8d1e-656bebceda7e None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:40.348663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9a87cf3e-ca09-4121-8d1e-656bebceda7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:40.421021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1127/2242] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:40.431479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:40.433350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:40.493614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4631e-f4ad-4a52-a787-6f03a85e14d5 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:40.498719 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26757c9-15a8-4d27-bb00-eaf094789ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:40.499358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26757c9-15a8-4d27-bb00-eaf094789ebc service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:40.500340 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26757c9-15a8-4d27-bb00-eaf094789ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:40.500623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26757c9-15a8-4d27-bb00-eaf094789ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:40.512649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26757c9-15a8-4d27-bb00-eaf094789ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:40.514163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26757c9-15a8-4d27-bb00-eaf094789ebc service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:40.525518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1116/2243] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:40.538139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db55777-b173-47ec-8f27-f5c690d189fe None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:40.538633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db55777-b173-47ec-8f27-f5c690d189fe None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:40.538726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db55777-b173-47ec-8f27-f5c690d189fe None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:40.540272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1128/2244] 10.222.0.85 () {54 vars in 912 bytes} [Mon Aug 29 20:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:28:40.546380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fdfb3-707c-456e-bb67-95811d3a0c3f None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:40.546681 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fdfb3-707c-456e-bb67-95811d3a0c3f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:40.547096 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fdfb3-707c-456e-bb67-95811d3a0c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:40.566750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-b20fdfb3-707c-456e-bb67-95811d3a0c3f None None] MFA Rules not processed for user `aa222b1c89dc4d2cb1686b4838c7908b`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:40.567934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b20fdfb3-707c-456e-bb67-95811d3a0c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:40.574926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1117/2245] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:28:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:40.586400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db55777-b173-47ec-8f27-f5c690d189fe None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:40.588727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5db55777-b173-47ec-8f27-f5c690d189fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:40.603354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db55777-b173-47ec-8f27-f5c690d189fe None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:40.608065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82023b-c16f-4e30-8203-0caae9c9ca3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:40.608723 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82023b-c16f-4e30-8203-0caae9c9ca3f service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:40.609270 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82023b-c16f-4e30-8203-0caae9c9ca3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:40.609738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82023b-c16f-4e30-8203-0caae9c9ca3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:40.618330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82023b-c16f-4e30-8203-0caae9c9ca3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:40.619986 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82023b-c16f-4e30-8203-0caae9c9ca3f service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:40.628811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1129/2246] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:41.064858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fdfb3-707c-456e-bb67-95811d3a0c3f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:41.067377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b20fdfb3-707c-456e-bb67-95811d3a0c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:41.085578 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fdfb3-707c-456e-bb67-95811d3a0c3f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:41.113348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830df0ac-a2bd-4e91-b9ef-d4c39555c967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:41.114000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df0ac-a2bd-4e91-b9ef-d4c39555c967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:41.114278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df0ac-a2bd-4e91-b9ef-d4c39555c967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:41.114550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df0ac-a2bd-4e91-b9ef-d4c39555c967 admin admin] PATH_INFO: `/v3/projects/99524f37a64d4ab68595c236ff02cc72` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:41.120204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830df0ac-a2bd-4e91-b9ef-d4c39555c967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99524f37a64d4ab68595c236ff02cc72)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:41.122581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830df0ac-a2bd-4e91-b9ef-d4c39555c967 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:41.281797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-830df0ac-a2bd-4e91-b9ef-d4c39555c967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99524f37a64d4ab68595c236ff02cc72', 'request_id': 'req-830df0ac-a2bd-4e91-b9ef-d4c39555c967'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:41.288448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-830df0ac-a2bd-4e91-b9ef-d4c39555c967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99524f37a64d4ab68595c236ff02cc72', 'request_id': 'req-830df0ac-a2bd-4e91-b9ef-d4c39555c967'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:41.388682 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-830df0ac-a2bd-4e91-b9ef-d4c39555c967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99524f37a64d4ab68595c236ff02cc72', 'request_id': 'req-830df0ac-a2bd-4e91-b9ef-d4c39555c967'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:41.400378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-830df0ac-a2bd-4e91-b9ef-d4c39555c967 admin admin] The token cache is being invalidate because project 99524f37a64d4ab68595c236ff02cc72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:41.400669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-830df0ac-a2bd-4e91-b9ef-d4c39555c967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:41.401302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-830df0ac-a2bd-4e91-b9ef-d4c39555c967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:41.402824 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1118/2247] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:41 2022] DELETE /identity/v3/projects/99524f37a64d4ab68595c236ff02cc72 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:41.407712 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df2bf6-ac2a-47d0-8344-d1af1f009875 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:41.407998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df2bf6-ac2a-47d0-8344-d1af1f009875 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:41.408338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df2bf6-ac2a-47d0-8344-d1af1f009875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:41.428785 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-68df2bf6-ac2a-47d0-8344-d1af1f009875 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:41.430045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-68df2bf6-ac2a-47d0-8344-d1af1f009875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:41.487585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1130/2248] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:41.495404 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c178f8e-4cdd-4aec-b7c2-c4a10e3c493c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:41.496298 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c178f8e-4cdd-4aec-b7c2-c4a10e3c493c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:41.496583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c178f8e-4cdd-4aec-b7c2-c4a10e3c493c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:41.519117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-6c178f8e-4cdd-4aec-b7c2-c4a10e3c493c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:41.520526 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-6c178f8e-4cdd-4aec-b7c2-c4a10e3c493c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:41.528569 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1119/2249] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:41.532865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df2bf6-ac2a-47d0-8344-d1af1f009875 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:41.542445 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df2bf6-ac2a-47d0-8344-d1af1f009875 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:41.546938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99968d-477e-4c48-8694-76f679694a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:41.547523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99968d-477e-4c48-8694-76f679694a0c admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:41.547796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99968d-477e-4c48-8694-76f679694a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:41.548063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99968d-477e-4c48-8694-76f679694a0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:41.548742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99968d-477e-4c48-8694-76f679694a0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:41.549066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99968d-477e-4c48-8694-76f679694a0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:41.550628 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99968d-477e-4c48-8694-76f679694a0c admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:41.566278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1131/2250] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:28:41.566742 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c178f8e-4cdd-4aec-b7c2-c4a10e3c493c None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:41.576411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c178f8e-4cdd-4aec-b7c2-c4a10e3c493c None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:41.581202 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea1db4-5a30-4618-b12c-2b01e1700e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:41.581835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea1db4-5a30-4618-b12c-2b01e1700e5c admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:41.582118 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea1db4-5a30-4618-b12c-2b01e1700e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:41.582392 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea1db4-5a30-4618-b12c-2b01e1700e5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:41.583081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea1db4-5a30-4618-b12c-2b01e1700e5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:41.585032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea1db4-5a30-4618-b12c-2b01e1700e5c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:41.762291 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1120/2251] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:41 2022] POST /identity/v3/projects => generated 375 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:41.766517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99968d-477e-4c48-8694-76f679694a0c admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:41.777298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99968d-477e-4c48-8694-76f679694a0c admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:41.832731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db4496-9999-4823-b0dc-8993a5fe652a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:41.833356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db4496-9999-4823-b0dc-8993a5fe652a admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:41.833643 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db4496-9999-4823-b0dc-8993a5fe652a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:41.833919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db4496-9999-4823-b0dc-8993a5fe652a admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:41.834559 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db4496-9999-4823-b0dc-8993a5fe652a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:41.837000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db4496-9999-4823-b0dc-8993a5fe652a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:42.005530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1132/2252] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:41 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:42.005963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea1db4-5a30-4618-b12c-2b01e1700e5c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:42.015861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea1db4-5a30-4618-b12c-2b01e1700e5c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:42.021140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30a531-3107-4b56-87f6-5ed306ed787a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:42.021870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30a531-3107-4b56-87f6-5ed306ed787a admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:42.022156 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30a531-3107-4b56-87f6-5ed306ed787a admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:42.022434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30a531-3107-4b56-87f6-5ed306ed787a admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:42.023086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30a531-3107-4b56-87f6-5ed306ed787a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:42.024746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30a531-3107-4b56-87f6-5ed306ed787a admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:42.035535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1121/2253] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:42.039525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db4496-9999-4823-b0dc-8993a5fe652a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:42.058131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db4496-9999-4823-b0dc-8993a5fe652a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:42.063261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7df46-3885-415a-be98-64fec5347b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:42.064291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7df46-3885-415a-be98-64fec5347b73 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:42.064519 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7df46-3885-415a-be98-64fec5347b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:42.064799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7df46-3885-415a-be98-64fec5347b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:42.065447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7df46-3885-415a-be98-64fec5347b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:42.067120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7df46-3885-415a-be98-64fec5347b73 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:42.079889 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1133/2254] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:42.087997 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30a531-3107-4b56-87f6-5ed306ed787a admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:42.098638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30a531-3107-4b56-87f6-5ed306ed787a admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:42.103427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a9b8a-2830-4dc6-bdae-72d9c140addc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:42.104080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a9b8a-2830-4dc6-bdae-72d9c140addc admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:42.104431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a9b8a-2830-4dc6-bdae-72d9c140addc admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:42.104708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a9b8a-2830-4dc6-bdae-72d9c140addc admin admin] PATH_INFO: `/v3/projects/931b2e61e51b4aadbcc7093c415168ec/users/1f12ff66dbc8473ea0a0a46864cdecfe/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:42.120084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a9b8a-2830-4dc6-bdae-72d9c140addc admin admin] RBAC: Authorizing `identity:create_grant(project_id=931b2e61e51b4aadbcc7093c415168ec, user_id=1f12ff66dbc8473ea0a0a46864cdecfe, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:42.122812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a9b8a-2830-4dc6-bdae-72d9c140addc admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:42.200914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1122/2255] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:42 2022] PUT /identity/v3/projects/931b2e61e51b4aadbcc7093c415168ec/users/1f12ff66dbc8473ea0a0a46864cdecfe/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:42.213285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0ddc8-cfa0-413d-a82b-8b6f869e7aa5 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:42.213571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0ddc8-cfa0-413d-a82b-8b6f869e7aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:42.217281 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0ddc8-cfa0-413d-a82b-8b6f869e7aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:42.252177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e6d0ddc8-cfa0-413d-a82b-8b6f869e7aa5 None None] MFA Rules not processed for user `1f12ff66dbc8473ea0a0a46864cdecfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:42.253419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e6d0ddc8-cfa0-413d-a82b-8b6f869e7aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:42.322173 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1134/2256] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:42.327262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a9b8a-2830-4dc6-bdae-72d9c140addc admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:42.392258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a9b8a-2830-4dc6-bdae-72d9c140addc admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:42.398306 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f25e4-d7d1-4039-8ba7-c43b17aad174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:42.398916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f25e4-d7d1-4039-8ba7-c43b17aad174 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:42.399219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f25e4-d7d1-4039-8ba7-c43b17aad174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:42.399478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f25e4-d7d1-4039-8ba7-c43b17aad174 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:42.400120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f25e4-d7d1-4039-8ba7-c43b17aad174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:42.402640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f25e4-d7d1-4039-8ba7-c43b17aad174 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:42.528713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:42.535766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1123/2257] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:42 2022] POST /identity/v3/projects => generated 375 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:42.540498 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0ddc8-cfa0-413d-a82b-8b6f869e7aa5 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:42.552455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0ddc8-cfa0-413d-a82b-8b6f869e7aa5 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:42.610441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084acdae-eaa9-4eaf-82ba-684fb92a3ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:42.611076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084acdae-eaa9-4eaf-82ba-684fb92a3ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:42.611401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084acdae-eaa9-4eaf-82ba-684fb92a3ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:42.611676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084acdae-eaa9-4eaf-82ba-684fb92a3ab7 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:42.612354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084acdae-eaa9-4eaf-82ba-684fb92a3ab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:42.614279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084acdae-eaa9-4eaf-82ba-684fb92a3ab7 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:42.700666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1135/2258] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:42 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:42.702394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f25e4-d7d1-4039-8ba7-c43b17aad174 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:42.713148 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f25e4-d7d1-4039-8ba7-c43b17aad174 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:42.718929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d76747-76f2-4e5e-8878-325564721c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:42.722441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d76747-76f2-4e5e-8878-325564721c09 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:42.722441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d76747-76f2-4e5e-8878-325564721c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:42.722441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d76747-76f2-4e5e-8878-325564721c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:42.722441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d76747-76f2-4e5e-8878-325564721c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:42.722441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d76747-76f2-4e5e-8878-325564721c09 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:42.734651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1124/2259] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:42.738637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084acdae-eaa9-4eaf-82ba-684fb92a3ab7 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:42.751200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084acdae-eaa9-4eaf-82ba-684fb92a3ab7 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:42.757280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ed122-8b84-4bad-9823-0372cb72a5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:42.758087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ed122-8b84-4bad-9823-0372cb72a5c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:42.758518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ed122-8b84-4bad-9823-0372cb72a5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:42.758861 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ed122-8b84-4bad-9823-0372cb72a5c8 admin admin] PATH_INFO: `/v3/projects/533274ab10844480b2d66a887765321b/users/ef524b55281f41869f275f7aa511653d/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:42.774651 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ed122-8b84-4bad-9823-0372cb72a5c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=533274ab10844480b2d66a887765321b, user_id=ef524b55281f41869f275f7aa511653d, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:42.777555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ed122-8b84-4bad-9823-0372cb72a5c8 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:42.800953 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:42.852441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1136/2260] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:42 2022] PUT /identity/v3/projects/533274ab10844480b2d66a887765321b/users/ef524b55281f41869f275f7aa511653d/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:42.856500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d76747-76f2-4e5e-8878-325564721c09 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:42.867193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d76747-76f2-4e5e-8878-325564721c09 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:42.935260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c58cdf-5b87-4b47-b195-69d7d0a1d92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:42.936053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c58cdf-5b87-4b47-b195-69d7d0a1d92e admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:42.936417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c58cdf-5b87-4b47-b195-69d7d0a1d92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:42.936786 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c58cdf-5b87-4b47-b195-69d7d0a1d92e admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:42.937490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c58cdf-5b87-4b47-b195-69d7d0a1d92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:42.939204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c58cdf-5b87-4b47-b195-69d7d0a1d92e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:42.952304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1125/2261] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:28:42.956741 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ed122-8b84-4bad-9823-0372cb72a5c8 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:42.968359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ed122-8b84-4bad-9823-0372cb72a5c8 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:42.973199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4af483-0ab1-4148-9392-656ba769ec37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:42.973856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4af483-0ab1-4148-9392-656ba769ec37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:42.974136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4af483-0ab1-4148-9392-656ba769ec37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:42.974411 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4af483-0ab1-4148-9392-656ba769ec37 admin admin] PATH_INFO: `/v3/projects/533274ab10844480b2d66a887765321b/users/ef524b55281f41869f275f7aa511653d/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:42.976408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4af483-0ab1-4148-9392-656ba769ec37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=533274ab10844480b2d66a887765321b, user_id=ef524b55281f41869f275f7aa511653d, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:42.979577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4af483-0ab1-4148-9392-656ba769ec37 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:42.990204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1137/2262] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:42 2022] PUT /identity/v3/projects/533274ab10844480b2d66a887765321b/users/ef524b55281f41869f275f7aa511653d/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:43.000743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d130c9-aef8-408e-87b7-01d7a814c343 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:43.001072 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d130c9-aef8-408e-87b7-01d7a814c343 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:43.001580 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d130c9-aef8-408e-87b7-01d7a814c343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:43.036824 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-c7d130c9-aef8-408e-87b7-01d7a814c343 None None] MFA Rules not processed for user `ef524b55281f41869f275f7aa511653d`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:43.038205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c7d130c9-aef8-408e-87b7-01d7a814c343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:43.135759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1126/2263] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:43.147586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4af483-0ab1-4148-9392-656ba769ec37 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:43.149354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-df4af483-0ab1-4148-9392-656ba769ec37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:43.205823 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4af483-0ab1-4148-9392-656ba769ec37 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:43.209504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69f999-d692-4de9-b096-c37de2dad56c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:43.210096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69f999-d692-4de9-b096-c37de2dad56c service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:43.210377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69f999-d692-4de9-b096-c37de2dad56c service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:43.210650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69f999-d692-4de9-b096-c37de2dad56c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:43.269149 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69f999-d692-4de9-b096-c37de2dad56c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:43.270618 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69f999-d692-4de9-b096-c37de2dad56c service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:43.280465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1138/2264] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:44.411741 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:45.034690 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:45.206161 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:45.223427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:45.508745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:45.531806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:45.732390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:45.754332 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:45.817685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:45.846285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:45.914645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:46.023620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:46.211542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:46.320352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:46.520630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:46.641191 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:46.732472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:47.195186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:47.920878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:48.557407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d130c9-aef8-408e-87b7-01d7a814c343 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:48.559853 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c7d130c9-aef8-408e-87b7-01d7a814c343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:48.612296 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:48.658853 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d130c9-aef8-408e-87b7-01d7a814c343 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:48.664765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:48.665551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:48.665902 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:48.666286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] PATH_INFO: `/v3/users/c060178cf4e5421bb794eee78bed9c9a` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:48.668402 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c060178cf4e5421bb794eee78bed9c9a)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:48.670424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:48.947102 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c060178cf4e5421bb794eee78bed9c9a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:48.948617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c060178cf4e5421bb794eee78bed9c9a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:48.957490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c060178cf4e5421bb794eee78bed9c9a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:49.057799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c060178cf4e5421bb794eee78bed9c9a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:49.059257 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c060178cf4e5421bb794eee78bed9c9a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:49.081957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1127/2265] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:48 2022] DELETE /identity/v3/users/c060178cf4e5421bb794eee78bed9c9a => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:49.091698 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29857ad6-176a-450e-8e8d-e3356bdb5f2c None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:49.092221 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29857ad6-176a-450e-8e8d-e3356bdb5f2c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:49.092315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29857ad6-176a-450e-8e8d-e3356bdb5f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:49.122767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-29857ad6-176a-450e-8e8d-e3356bdb5f2c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:49.124135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-29857ad6-176a-450e-8e8d-e3356bdb5f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:49.166679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:49.169113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:49.231210 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1139/2266] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:49.247129 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29857ad6-176a-450e-8e8d-e3356bdb5f2c None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:49.248964 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-29857ad6-176a-450e-8e8d-e3356bdb5f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:49.315950 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29857ad6-176a-450e-8e8d-e3356bdb5f2c None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:49.320030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec9e8e-848b-4111-8216-73007ca0039d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:49.320824 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9e8e-848b-4111-8216-73007ca0039d service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:49.326232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9e8e-848b-4111-8216-73007ca0039d service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:49.326232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9e8e-848b-4111-8216-73007ca0039d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:49.326447 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40213601-dacb-40b1-b38c-b5f0a9923a8e admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:49.332145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec9e8e-848b-4111-8216-73007ca0039d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:49.332307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:49.332941 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:49.333234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:49.333529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] PATH_INFO: `/v3/users/1f12ff66dbc8473ea0a0a46864cdecfe` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:49.334048 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec9e8e-848b-4111-8216-73007ca0039d service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:49.334822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f12ff66dbc8473ea0a0a46864cdecfe)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:49.337082 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:49.345058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1140/2267] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:28:49.406883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:49.640484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f12ff66dbc8473ea0a0a46864cdecfe'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:49.641622 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f12ff66dbc8473ea0a0a46864cdecfe'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:49.649222 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f12ff66dbc8473ea0a0a46864cdecfe'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:49.788669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f12ff66dbc8473ea0a0a46864cdecfe'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:49.789289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f12ff66dbc8473ea0a0a46864cdecfe'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:49.808075 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1128/2268] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:49 2022] DELETE /identity/v3/users/1f12ff66dbc8473ea0a0a46864cdecfe => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:49.812328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e67b9-1a25-4daa-aed8-1e7d4b81e315 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:49.812613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e67b9-1a25-4daa-aed8-1e7d4b81e315 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:49.812887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e67b9-1a25-4daa-aed8-1e7d4b81e315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:49.832951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-dc1e67b9-1a25-4daa-aed8-1e7d4b81e315 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:49.834093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-dc1e67b9-1a25-4daa-aed8-1e7d4b81e315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:49.880205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:49.881859 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:49.918884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1141/2269] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:49.928910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e67b9-1a25-4daa-aed8-1e7d4b81e315 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:49.930926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-dc1e67b9-1a25-4daa-aed8-1e7d4b81e315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:49.962055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235b821-11b0-40d3-a643-c91a2fe3d0f4 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:49.969331 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:49.970371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:49.970671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:49.971019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] PATH_INFO: `/v3/projects/b127f4344e32491b9d7d3bf4abb0f527` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:49.972218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b127f4344e32491b9d7d3bf4abb0f527)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:49.974152 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:49.997867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e67b9-1a25-4daa-aed8-1e7d4b81e315 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:50.002341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa5ab8-a509-4bca-afcb-c4f6b9562379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:50.002963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa5ab8-a509-4bca-afcb-c4f6b9562379 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:50.003250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa5ab8-a509-4bca-afcb-c4f6b9562379 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:50.003526 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa5ab8-a509-4bca-afcb-c4f6b9562379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:50.013569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa5ab8-a509-4bca-afcb-c4f6b9562379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:50.015008 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa5ab8-a509-4bca-afcb-c4f6b9562379 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:50.025338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1142/2270] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:28:50.140388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b127f4344e32491b9d7d3bf4abb0f527', 'request_id': 'req-d72a170a-6d39-4d69-addc-79d58651b5c4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.147456 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b127f4344e32491b9d7d3bf4abb0f527', 'request_id': 'req-d72a170a-6d39-4d69-addc-79d58651b5c4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.165217 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa5ab8-a509-4bca-afcb-c4f6b9562379 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:50.167616 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-14fa5ab8-a509-4bca-afcb-c4f6b9562379 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:50.180276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa5ab8-a509-4bca-afcb-c4f6b9562379 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:50.237582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:50.238218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:50.238519 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:50.238833 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] PATH_INFO: `/v3/users/8601cac79b394be9bf48df3ef1ae89a6` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:50.240013 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8601cac79b394be9bf48df3ef1ae89a6)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:50.242331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:50.269973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b127f4344e32491b9d7d3bf4abb0f527', 'request_id': 'req-d72a170a-6d39-4d69-addc-79d58651b5c4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.283909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] The token cache is being invalidate because project b127f4344e32491b9d7d3bf4abb0f527 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:50.284347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.285141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.286980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1129/2271] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:49 2022] DELETE /identity/v3/projects/b127f4344e32491b9d7d3bf4abb0f527 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:50.290658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:50.293370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:50.309272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a170a-6d39-4d69-addc-79d58651b5c4 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:50.319260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c2430-a3b3-4a21-9468-4990c7cb4bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:50.320304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c2430-a3b3-4a21-9468-4990c7cb4bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:50.320686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c2430-a3b3-4a21-9468-4990c7cb4bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:50.321200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c2430-a3b3-4a21-9468-4990c7cb4bea admin admin] PATH_INFO: `/v3/users/50b02dcb74f14237acb66629a88d894e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:50.322480 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c2430-a3b3-4a21-9468-4990c7cb4bea admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b02dcb74f14237acb66629a88d894e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:50.326101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c2430-a3b3-4a21-9468-4990c7cb4bea admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:50.395243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:50.563282 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8601cac79b394be9bf48df3ef1ae89a6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.570276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8601cac79b394be9bf48df3ef1ae89a6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.591147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3b1c2430-a3b3-4a21-9468-4990c7cb4bea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b02dcb74f14237acb66629a88d894e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.591764 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3b1c2430-a3b3-4a21-9468-4990c7cb4bea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b02dcb74f14237acb66629a88d894e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.598777 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3b1c2430-a3b3-4a21-9468-4990c7cb4bea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b02dcb74f14237acb66629a88d894e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.635192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:50.686444 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8601cac79b394be9bf48df3ef1ae89a6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.687228 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8601cac79b394be9bf48df3ef1ae89a6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.701419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8601cac79b394be9bf48df3ef1ae89a6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.708069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1143/2272] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:50 2022] DELETE /identity/v3/users/8601cac79b394be9bf48df3ef1ae89a6 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:50.709266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:50.711918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:50.742942 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3b1c2430-a3b3-4a21-9468-4990c7cb4bea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b02dcb74f14237acb66629a88d894e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.743791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3b1c2430-a3b3-4a21-9468-4990c7cb4bea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b02dcb74f14237acb66629a88d894e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:50.763894 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1130/2273] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:50 2022] DELETE /identity/v3/users/50b02dcb74f14237acb66629a88d894e => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:50.766192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c54f6-77a3-440a-856e-749c330fe270 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:50.766702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c54f6-77a3-440a-856e-749c330fe270 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:50.767193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c54f6-77a3-440a-856e-749c330fe270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:50.791823 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-e85c54f6-77a3-440a-856e-749c330fe270 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:50.793621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e85c54f6-77a3-440a-856e-749c330fe270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:50.796457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45826be-4102-4192-a184-c8901aebd919 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:50.868534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:50.868874 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1131/2274] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:50.869731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:50.870085 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:50.870472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] PATH_INFO: `/v3/projects/931b2e61e51b4aadbcc7093c415168ec` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:50.871762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=931b2e61e51b4aadbcc7093c415168ec)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:50.873888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:50.889511 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c54f6-77a3-440a-856e-749c330fe270 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:50.894648 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e85c54f6-77a3-440a-856e-749c330fe270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:50.959383 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c54f6-77a3-440a-856e-749c330fe270 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:50.963078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcccae4-e9e6-49e9-9320-f45385064bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:50.963687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcccae4-e9e6-49e9-9320-f45385064bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:50.963969 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcccae4-e9e6-49e9-9320-f45385064bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:50.964312 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcccae4-e9e6-49e9-9320-f45385064bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:50.973562 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcccae4-e9e6-49e9-9320-f45385064bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:50.975009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcccae4-e9e6-49e9-9320-f45385064bc8 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:50.985220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1132/2275] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:51.047988 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '931b2e61e51b4aadbcc7093c415168ec', 'request_id': 'req-e62dac9e-5142-4ff3-8454-a82707ecebb0'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.055332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '931b2e61e51b4aadbcc7093c415168ec', 'request_id': 'req-e62dac9e-5142-4ff3-8454-a82707ecebb0'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.062414 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '931b2e61e51b4aadbcc7093c415168ec', 'request_id': 'req-e62dac9e-5142-4ff3-8454-a82707ecebb0'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.180493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcccae4-e9e6-49e9-9320-f45385064bc8 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:51.182038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9bcccae4-e9e6-49e9-9320-f45385064bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:51.188809 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] The token cache is being invalidate because project 931b2e61e51b4aadbcc7093c415168ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:51.189298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.189936 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.192183 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1144/2276] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:50 2022] DELETE /identity/v3/projects/931b2e61e51b4aadbcc7093c415168ec => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:51.197006 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcccae4-e9e6-49e9-9320-f45385064bc8 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:51.197963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:51.199423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:51.304028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:51.304770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:51.305109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:51.305593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] PATH_INFO: `/v3/users/f8dbf240ad1743c7ac152ce8ddd29213` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:51.306532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dac9e-5142-4ff3-8454-a82707ecebb0 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:51.306886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8dbf240ad1743c7ac152ce8ddd29213)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:51.309040 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:51.312551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:51.313318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:51.313609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:51.313954 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] PATH_INFO: `/v3/users/ef524b55281f41869f275f7aa511653d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:51.315184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef524b55281f41869f275f7aa511653d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:51.318699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:51.532532 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef524b55281f41869f275f7aa511653d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.541015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8dbf240ad1743c7ac152ce8ddd29213'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.541810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8dbf240ad1743c7ac152ce8ddd29213'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.551134 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8dbf240ad1743c7ac152ce8ddd29213'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.556072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef524b55281f41869f275f7aa511653d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.605835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8dbf240ad1743c7ac152ce8ddd29213'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.606522 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8dbf240ad1743c7ac152ce8ddd29213'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.626548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1133/2277] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:51 2022] DELETE /identity/v3/users/f8dbf240ad1743c7ac152ce8ddd29213 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:51.627722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:51.629193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:51.670030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef524b55281f41869f275f7aa511653d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.671067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef524b55281f41869f275f7aa511653d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.684680 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef524b55281f41869f275f7aa511653d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.695025 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1145/2278] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:51 2022] DELETE /identity/v3/users/ef524b55281f41869f275f7aa511653d => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:51.696669 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:51.698341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:51.703122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5809e-8c02-4049-9af6-bcaf0cdc0a8d admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:51.772045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e1c99-1648-4e30-9cc6-afff3f7ae451 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:51.778154 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122b750-b318-4d82-a045-4071fbb91787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:51.779009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122b750-b318-4d82-a045-4071fbb91787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:51.779476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122b750-b318-4d82-a045-4071fbb91787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:51.779960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122b750-b318-4d82-a045-4071fbb91787 admin admin] PATH_INFO: `/v3/projects/66fd885b97fb46ce9b8a1fd002a2757b` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:51.781368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122b750-b318-4d82-a045-4071fbb91787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66fd885b97fb46ce9b8a1fd002a2757b)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:51.783289 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122b750-b318-4d82-a045-4071fbb91787 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:51.798806 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:51.800300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:51.800587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:51.800862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] PATH_INFO: `/v3/projects/d0fe989af30c4e15b96833e9773b6a94` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:51.802015 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0fe989af30c4e15b96833e9773b6a94)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:51.805008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:51.912995 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9122b750-b318-4d82-a045-4071fbb91787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66fd885b97fb46ce9b8a1fd002a2757b', 'request_id': 'req-9122b750-b318-4d82-a045-4071fbb91787'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.920583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9122b750-b318-4d82-a045-4071fbb91787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66fd885b97fb46ce9b8a1fd002a2757b', 'request_id': 'req-9122b750-b318-4d82-a045-4071fbb91787'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:51.927380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9122b750-b318-4d82-a045-4071fbb91787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66fd885b97fb46ce9b8a1fd002a2757b', 'request_id': 'req-9122b750-b318-4d82-a045-4071fbb91787'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.039883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9122b750-b318-4d82-a045-4071fbb91787 admin admin] The token cache is being invalidate because project 66fd885b97fb46ce9b8a1fd002a2757b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:52.040230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9122b750-b318-4d82-a045-4071fbb91787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.041045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9122b750-b318-4d82-a045-4071fbb91787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.043180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1146/2279] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:51 2022] DELETE /identity/v3/projects/66fd885b97fb46ce9b8a1fd002a2757b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:52.057784 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15921c-e4b7-4cf1-9af3-c558c2834c56 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:52.058146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15921c-e4b7-4cf1-9af3-c558c2834c56 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:52.058432 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15921c-e4b7-4cf1-9af3-c558c2834c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:52.087990 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-db15921c-e4b7-4cf1-9af3-c558c2834c56 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:52.089995 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-db15921c-e4b7-4cf1-9af3-c558c2834c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:52.151301 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0fe989af30c4e15b96833e9773b6a94', 'request_id': 'req-732af3bc-66e0-4eb5-9970-e355aff2593b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.159351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0fe989af30c4e15b96833e9773b6a94', 'request_id': 'req-732af3bc-66e0-4eb5-9970-e355aff2593b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.181020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1147/2280] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:52.181906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15921c-e4b7-4cf1-9af3-c558c2834c56 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:52.185390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-db15921c-e4b7-4cf1-9af3-c558c2834c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:52.200800 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15921c-e4b7-4cf1-9af3-c558c2834c56 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:52.259915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21355b68-0327-49c2-8405-599c26d4c4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:52.260604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21355b68-0327-49c2-8405-599c26d4c4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:52.260892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21355b68-0327-49c2-8405-599c26d4c4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:52.261834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21355b68-0327-49c2-8405-599c26d4c4b0 admin admin] PATH_INFO: `/v3/users/c2abe65262604d89adc92ff86b1748c1` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:52.262977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21355b68-0327-49c2-8405-599c26d4c4b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2abe65262604d89adc92ff86b1748c1)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:52.265064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21355b68-0327-49c2-8405-599c26d4c4b0 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:52.314616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0fe989af30c4e15b96833e9773b6a94', 'request_id': 'req-732af3bc-66e0-4eb5-9970-e355aff2593b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.326868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] The token cache is being invalidate because project d0fe989af30c4e15b96833e9773b6a94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:52.327163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.327802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.329318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1134/2281] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:51 2022] DELETE /identity/v3/projects/d0fe989af30c4e15b96833e9773b6a94 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:52.330079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:52.331344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:52.389581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732af3bc-66e0-4eb5-9970-e355aff2593b admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:52.393074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ba087-1f52-4726-b1c8-1af8c4954c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:52.394070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ba087-1f52-4726-b1c8-1af8c4954c1d service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:52.394347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ba087-1f52-4726-b1c8-1af8c4954c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:52.394617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ba087-1f52-4726-b1c8-1af8c4954c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:52.396374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7a3ba087-1f52-4726-b1c8-1af8c4954c1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:52.408246 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ba087-1f52-4726-b1c8-1af8c4954c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:52.409704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ba087-1f52-4726-b1c8-1af8c4954c1d service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:52.419565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1135/2282] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:52.420373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ba087-1f52-4726-b1c8-1af8c4954c1d service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:52.421638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7a3ba087-1f52-4726-b1c8-1af8c4954c1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:52.430865 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:52.435247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ba087-1f52-4726-b1c8-1af8c4954c1d service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:52.439940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad2b25-b154-4784-8cc2-6e1baa30d184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:52.440843 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad2b25-b154-4784-8cc2-6e1baa30d184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:52.441139 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad2b25-b154-4784-8cc2-6e1baa30d184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:52.441411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad2b25-b154-4784-8cc2-6e1baa30d184 admin admin] PATH_INFO: `/v3/projects/533274ab10844480b2d66a887765321b` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:52.442572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad2b25-b154-4784-8cc2-6e1baa30d184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=533274ab10844480b2d66a887765321b)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:52.444468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad2b25-b154-4784-8cc2-6e1baa30d184 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:52.472327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-21355b68-0327-49c2-8405-599c26d4c4b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2abe65262604d89adc92ff86b1748c1'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.480381 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-21355b68-0327-49c2-8405-599c26d4c4b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2abe65262604d89adc92ff86b1748c1'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.563304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d7ad2b25-b154-4784-8cc2-6e1baa30d184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '533274ab10844480b2d66a887765321b', 'request_id': 'req-d7ad2b25-b154-4784-8cc2-6e1baa30d184'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.570839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d7ad2b25-b154-4784-8cc2-6e1baa30d184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '533274ab10844480b2d66a887765321b', 'request_id': 'req-d7ad2b25-b154-4784-8cc2-6e1baa30d184'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.572245 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:52.573056 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-21355b68-0327-49c2-8405-599c26d4c4b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2abe65262604d89adc92ff86b1748c1'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.573896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-21355b68-0327-49c2-8405-599c26d4c4b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2abe65262604d89adc92ff86b1748c1'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.586563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-21355b68-0327-49c2-8405-599c26d4c4b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2abe65262604d89adc92ff86b1748c1'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.593908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1148/2283] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:52 2022] DELETE /identity/v3/users/c2abe65262604d89adc92ff86b1748c1 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:52.595205 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce8792-1ca4-421e-8d75-c852d0bb69f9 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:52.595524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce8792-1ca4-421e-8d75-c852d0bb69f9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:52.595792 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce8792-1ca4-421e-8d75-c852d0bb69f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:52.616276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-84ce8792-1ca4-421e-8d75-c852d0bb69f9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:52.617464 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-84ce8792-1ca4-421e-8d75-c852d0bb69f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:52.657609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d7ad2b25-b154-4784-8cc2-6e1baa30d184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '533274ab10844480b2d66a887765321b', 'request_id': 'req-d7ad2b25-b154-4784-8cc2-6e1baa30d184'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.671928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d7ad2b25-b154-4784-8cc2-6e1baa30d184 admin admin] The token cache is being invalidate because project 533274ab10844480b2d66a887765321b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:52.672285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d7ad2b25-b154-4784-8cc2-6e1baa30d184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.672958 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d7ad2b25-b154-4784-8cc2-6e1baa30d184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:52.675022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1136/2284] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:52 2022] DELETE /identity/v3/projects/533274ab10844480b2d66a887765321b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:52.679966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2836d-57ca-4bbd-adb0-8abea841f744 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:52.680280 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2836d-57ca-4bbd-adb0-8abea841f744 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:52.680558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2836d-57ca-4bbd-adb0-8abea841f744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:52.680843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1149/2285] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:52.691193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6740dfa-7598-47aa-99ba-ae3f988581f6 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:52.691557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6740dfa-7598-47aa-99ba-ae3f988581f6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:52.691759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6740dfa-7598-47aa-99ba-ae3f988581f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:52.712571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e6740dfa-7598-47aa-99ba-ae3f988581f6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:52.713978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e6740dfa-7598-47aa-99ba-ae3f988581f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:52.722343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-29e2836d-57ca-4bbd-adb0-8abea841f744 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:52.724217 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1150/2286] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:52.724388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-29e2836d-57ca-4bbd-adb0-8abea841f744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:52.727345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6740dfa-7598-47aa-99ba-ae3f988581f6 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:52.736507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6740dfa-7598-47aa-99ba-ae3f988581f6 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:52.740890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1137/2287] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:28:52.742176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bc543-7624-444f-a179-d5a731d90b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:52.742797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bc543-7624-444f-a179-d5a731d90b75 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:52.742904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd04028-868d-494b-808d-028e65b1ebf9 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:52.743114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bc543-7624-444f-a179-d5a731d90b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:52.743217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd04028-868d-494b-808d-028e65b1ebf9 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:52.743387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bc543-7624-444f-a179-d5a731d90b75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:52.743488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd04028-868d-494b-808d-028e65b1ebf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:52.744064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76bc543-7624-444f-a179-d5a731d90b75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:52.744768 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76bc543-7624-444f-a179-d5a731d90b75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:52.747192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76bc543-7624-444f-a179-d5a731d90b75 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:52.767131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1151/2288] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:28:52.767324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-ebd04028-868d-494b-808d-028e65b1ebf9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:52.767324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ebd04028-868d-494b-808d-028e65b1ebf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:52.768254 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bc543-7624-444f-a179-d5a731d90b75 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:52.774585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1138/2289] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:52.778871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd04028-868d-494b-808d-028e65b1ebf9 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:52.784121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bc543-7624-444f-a179-d5a731d90b75 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:52.788996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd04028-868d-494b-808d-028e65b1ebf9 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:52.789148 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0ecea-4ad9-44dc-83d9-721685ada690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:52.789727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0ecea-4ad9-44dc-83d9-721685ada690 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:52.790206 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0ecea-4ad9-44dc-83d9-721685ada690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:52.790745 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0ecea-4ad9-44dc-83d9-721685ada690 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:52.791551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0ecea-4ad9-44dc-83d9-721685ada690 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:52.793817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0ecea-4ad9-44dc-83d9-721685ada690 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:52.794906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79279c2b-19e3-4e0f-bf05-1f4a63870da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:52.795509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79279c2b-19e3-4e0f-bf05-1f4a63870da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:52.798810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79279c2b-19e3-4e0f-bf05-1f4a63870da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:52.798810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79279c2b-19e3-4e0f-bf05-1f4a63870da4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:52.798810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79279c2b-19e3-4e0f-bf05-1f4a63870da4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:52.798810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79279c2b-19e3-4e0f-bf05-1f4a63870da4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:52.799532 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79279c2b-19e3-4e0f-bf05-1f4a63870da4 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:52.815701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1139/2290] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:28:52.816885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79279c2b-19e3-4e0f-bf05-1f4a63870da4 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:52.827046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79279c2b-19e3-4e0f-bf05-1f4a63870da4 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:52.833420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8649b-02d8-488e-b2ae-db4ad700087c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:52.833420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8649b-02d8-488e-b2ae-db4ad700087c admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:52.833420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8649b-02d8-488e-b2ae-db4ad700087c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:52.833420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8649b-02d8-488e-b2ae-db4ad700087c admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:52.834236 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8649b-02d8-488e-b2ae-db4ad700087c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:52.835772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8649b-02d8-488e-b2ae-db4ad700087c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:52.889121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1152/2291] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:52 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:28:52.893088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0ecea-4ad9-44dc-83d9-721685ada690 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:52.960816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0ecea-4ad9-44dc-83d9-721685ada690 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:52.966319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b987ef2-85e9-41c3-9562-36b360228295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:52.966985 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b987ef2-85e9-41c3-9562-36b360228295 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:52.967261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b987ef2-85e9-41c3-9562-36b360228295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:52.967717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b987ef2-85e9-41c3-9562-36b360228295 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:52.968884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b987ef2-85e9-41c3-9562-36b360228295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:52.972691 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1140/2292] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:52 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:52.972825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b987ef2-85e9-41c3-9562-36b360228295 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:52.974181 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8649b-02d8-488e-b2ae-db4ad700087c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:53.054466 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8649b-02d8-488e-b2ae-db4ad700087c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:53.058644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1153/2293] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:52 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:53.059565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b987ef2-85e9-41c3-9562-36b360228295 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:53.060532 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15ad5e-dbc6-462c-9255-72d9d59a03f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:53.061120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7b987ef2-85e9-41c3-9562-36b360228295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:53.061300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15ad5e-dbc6-462c-9255-72d9d59a03f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:53.061607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15ad5e-dbc6-462c-9255-72d9d59a03f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:53.061998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15ad5e-dbc6-462c-9255-72d9d59a03f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:53.062704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15ad5e-dbc6-462c-9255-72d9d59a03f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:53.064834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15ad5e-dbc6-462c-9255-72d9d59a03f6 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:53.074739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b987ef2-85e9-41c3-9562-36b360228295 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:53.081431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:53.082009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:53.082285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:53.082570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] PATH_INFO: `/v3/projects/6557d6a51ff94049be43d40321c09526` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:53.083659 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6557d6a51ff94049be43d40321c09526)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:53.085609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:53.154558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1141/2294] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:52 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:53.155545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15ad5e-dbc6-462c-9255-72d9d59a03f6 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:53.157380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-fa15ad5e-dbc6-462c-9255-72d9d59a03f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:53.181081 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15ad5e-dbc6-462c-9255-72d9d59a03f6 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:53.194274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6557d6a51ff94049be43d40321c09526', 'request_id': 'req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.202405 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6557d6a51ff94049be43d40321c09526', 'request_id': 'req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.211292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6557d6a51ff94049be43d40321c09526', 'request_id': 'req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.247564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:53.248240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:53.248551 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:53.248797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] PATH_INFO: `/v3/projects/e19c5148270a4b7697391ea799683636` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:53.249939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19c5148270a4b7697391ea799683636)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:53.251907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:53.302980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] The token cache is being invalidate because project 6557d6a51ff94049be43d40321c09526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:53.303327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.303921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.308670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1154/2295] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:53 2022] DELETE /identity/v3/projects/6557d6a51ff94049be43d40321c09526 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:53.308670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:53.310772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:53.324919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05f8b6-417f-4d4b-a0df-b501e616e4bb admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:53.331044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156019d-3a2f-49a5-8347-bc04d69c9836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:53.331744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156019d-3a2f-49a5-8347-bc04d69c9836 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:53.332614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156019d-3a2f-49a5-8347-bc04d69c9836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:53.332614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156019d-3a2f-49a5-8347-bc04d69c9836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:53.333151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156019d-3a2f-49a5-8347-bc04d69c9836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:53.334806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156019d-3a2f-49a5-8347-bc04d69c9836 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:53.349121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1155/2296] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:53.349121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156019d-3a2f-49a5-8347-bc04d69c9836 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:53.349121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7156019d-3a2f-49a5-8347-bc04d69c9836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:53.364231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156019d-3a2f-49a5-8347-bc04d69c9836 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:53.379196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19c5148270a4b7697391ea799683636', 'request_id': 'req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.384752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19c5148270a4b7697391ea799683636', 'request_id': 'req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.422066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b4525-29dc-4121-a313-54270b4f4bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:53.422674 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b4525-29dc-4121-a313-54270b4f4bff admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:53.422955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b4525-29dc-4121-a313-54270b4f4bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:53.423233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b4525-29dc-4121-a313-54270b4f4bff admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:53.423906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b4525-29dc-4121-a313-54270b4f4bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:53.425546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b4525-29dc-4121-a313-54270b4f4bff admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:53.435381 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1156/2297] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:53.436098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b4525-29dc-4121-a313-54270b4f4bff admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:53.437740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-923b4525-29dc-4121-a313-54270b4f4bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:53.449993 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b4525-29dc-4121-a313-54270b4f4bff admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:53.454586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:53.455171 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:53.455454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:53.455767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] PATH_INFO: `/v3/users/3c8766948d6845cb89a7be69a8a88601` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:53.456923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c8766948d6845cb89a7be69a8a88601)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:53.458792 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:53.475313 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19c5148270a4b7697391ea799683636', 'request_id': 'req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.488598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] The token cache is being invalidate because project e19c5148270a4b7697391ea799683636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:53.488888 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.489713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.491979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1142/2298] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:53 2022] DELETE /identity/v3/projects/e19c5148270a4b7697391ea799683636 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:53.492104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:53.493446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:53.506082 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c51a7-d6cd-4ba7-8e51-5b01ed33c683 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:53.510835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a135a7-2ab5-4636-9df3-9fb12bf90c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:53.511460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a135a7-2ab5-4636-9df3-9fb12bf90c1f admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:53.511744 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a135a7-2ab5-4636-9df3-9fb12bf90c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:53.512021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a135a7-2ab5-4636-9df3-9fb12bf90c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:53.512807 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a135a7-2ab5-4636-9df3-9fb12bf90c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:53.514818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a135a7-2ab5-4636-9df3-9fb12bf90c1f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:53.525772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1143/2299] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:53.527122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a135a7-2ab5-4636-9df3-9fb12bf90c1f admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:53.529893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-17a135a7-2ab5-4636-9df3-9fb12bf90c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:53.543256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a135a7-2ab5-4636-9df3-9fb12bf90c1f admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:53.547984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f33ea-1875-492f-ad74-1ed3be4137d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:53.548663 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f33ea-1875-492f-ad74-1ed3be4137d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:53.548941 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f33ea-1875-492f-ad74-1ed3be4137d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:53.549213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f33ea-1875-492f-ad74-1ed3be4137d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:53.549843 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f33ea-1875-492f-ad74-1ed3be4137d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:53.551421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f33ea-1875-492f-ad74-1ed3be4137d6 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:53.561414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1144/2300] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:53.562223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f33ea-1875-492f-ad74-1ed3be4137d6 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:53.563846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a10f33ea-1875-492f-ad74-1ed3be4137d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:53.577164 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f33ea-1875-492f-ad74-1ed3be4137d6 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:53.582009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:53.582637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:53.582902 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:53.583180 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] PATH_INFO: `/v3/users/3d8b4954e24a498fb12bf938f74b29c5` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:53.584377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d8b4954e24a498fb12bf938f74b29c5)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:53.586293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:53.653958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c8766948d6845cb89a7be69a8a88601'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.660750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c8766948d6845cb89a7be69a8a88601'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.759847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c8766948d6845cb89a7be69a8a88601'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.760440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c8766948d6845cb89a7be69a8a88601'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.773701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c8766948d6845cb89a7be69a8a88601'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.780123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d8b4954e24a498fb12bf938f74b29c5'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.780753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1157/2301] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:53 2022] DELETE /identity/v3/users/3c8766948d6845cb89a7be69a8a88601 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:53.780881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d8b4954e24a498fb12bf938f74b29c5'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.781692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:53.783232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:53.788219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d8b4954e24a498fb12bf938f74b29c5'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.829017 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:53.857873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d686b-adff-4a4e-b984-c17c3d50a5ad admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:53.862814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbf75f-5858-4cda-ae4c-715ed8ef2f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:53.863481 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbf75f-5858-4cda-ae4c-715ed8ef2f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:53.863931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbf75f-5858-4cda-ae4c-715ed8ef2f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:53.864244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbf75f-5858-4cda-ae4c-715ed8ef2f59 admin admin] PATH_INFO: `/v3/projects/4d852297e12a4b1c942d6661b3edd97a/users/2eb79017317e43189f662d4fe8d3f16e/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:53.874662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d8b4954e24a498fb12bf938f74b29c5'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.875517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d8b4954e24a498fb12bf938f74b29c5'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:53.886620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbf75f-5858-4cda-ae4c-715ed8ef2f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d852297e12a4b1c942d6661b3edd97a, user_id=2eb79017317e43189f662d4fe8d3f16e, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:53.890477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbf75f-5858-4cda-ae4c-715ed8ef2f59 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:53.895009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1145/2302] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:28:53 2022] DELETE /identity/v3/users/3d8b4954e24a498fb12bf938f74b29c5 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:53.896070 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:53.897759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:53.968544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1158/2303] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:53 2022] PUT /identity/v3/projects/4d852297e12a4b1c942d6661b3edd97a/users/2eb79017317e43189f662d4fe8d3f16e/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:53.976444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6f934-7270-4248-bf67-3b4422d00e31 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:53.980336 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d8b13-cbee-4a17-9dfc-e995aad67488 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:53.980888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d8b13-cbee-4a17-9dfc-e995aad67488 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:53.981367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d8b13-cbee-4a17-9dfc-e995aad67488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.019232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-980d8b13-cbee-4a17-9dfc-e995aad67488 None None] MFA Rules not processed for user `2eb79017317e43189f662d4fe8d3f16e`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:54.020823 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-980d8b13-cbee-4a17-9dfc-e995aad67488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:54.042910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5feb7fd-7013-4c15-9edd-bd7cd223e3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:54.043737 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5feb7fd-7013-4c15-9edd-bd7cd223e3fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.044125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5feb7fd-7013-4c15-9edd-bd7cd223e3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.044543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5feb7fd-7013-4c15-9edd-bd7cd223e3fb admin admin] PATH_INFO: `/v3/projects/8c28205e89c1441bbb3f1d68ac8893e4/users/95d668bfaf71417baac3c4336c33ed9f/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.065215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5feb7fd-7013-4c15-9edd-bd7cd223e3fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c28205e89c1441bbb3f1d68ac8893e4, user_id=95d668bfaf71417baac3c4336c33ed9f, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:54.068437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5feb7fd-7013-4c15-9edd-bd7cd223e3fb admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:54.114279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1159/2304] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:54.115214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d8b13-cbee-4a17-9dfc-e995aad67488 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:54.116546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-980d8b13-cbee-4a17-9dfc-e995aad67488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:54.130759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d8b13-cbee-4a17-9dfc-e995aad67488 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:54.135430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:54.136847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.137137 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.137406 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97 admin admin] PATH_INFO: `/v3/projects/9cba124a13c44703a0625f8d18cb5037` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.138543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cba124a13c44703a0625f8d18cb5037)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:54.140518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:54.200832 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:54.206723 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1146/2305] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:53 2022] PUT /identity/v3/projects/8c28205e89c1441bbb3f1d68ac8893e4/users/95d668bfaf71417baac3c4336c33ed9f/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:54.209002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5feb7fd-7013-4c15-9edd-bd7cd223e3fb admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:54.210921 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e5feb7fd-7013-4c15-9edd-bd7cd223e3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:54.273996 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cba124a13c44703a0625f8d18cb5037', 'request_id': 'req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:54.285705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cba124a13c44703a0625f8d18cb5037', 'request_id': 'req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:54.286652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5feb7fd-7013-4c15-9edd-bd7cd223e3fb admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:54.294502 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cba124a13c44703a0625f8d18cb5037', 'request_id': 'req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:54.345554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d68d8e-b59e-4ffc-a932-008999ee72c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:54.346163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d68d8e-b59e-4ffc-a932-008999ee72c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.346440 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d68d8e-b59e-4ffc-a932-008999ee72c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.346708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d68d8e-b59e-4ffc-a932-008999ee72c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.354399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d68d8e-b59e-4ffc-a932-008999ee72c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:54.354399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d68d8e-b59e-4ffc-a932-008999ee72c9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:54.373310 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97 admin admin] The token cache is being invalidate because project 9cba124a13c44703a0625f8d18cb5037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:54.373789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:54.374476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b4a0e17c-fce0-4fe5-a62a-495b96b74d97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:54.376415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1160/2306] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:54 2022] DELETE /identity/v3/projects/9cba124a13c44703a0625f8d18cb5037 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:54.377933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d159d-d6d1-4cb1-911a-87883a7954d7 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.382677 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d159d-d6d1-4cb1-911a-87883a7954d7 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.382677 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d159d-d6d1-4cb1-911a-87883a7954d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.402925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b89d159d-d6d1-4cb1-911a-87883a7954d7 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:54.404144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b89d159d-d6d1-4cb1-911a-87883a7954d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:54.411756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1161/2307] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:54.413094 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df5dd0-0a3d-4f77-b7a8-8ed0247a2a71 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.413372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df5dd0-0a3d-4f77-b7a8-8ed0247a2a71 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.413643 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df5dd0-0a3d-4f77-b7a8-8ed0247a2a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.436700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-42df5dd0-0a3d-4f77-b7a8-8ed0247a2a71 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:54.437906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-42df5dd0-0a3d-4f77-b7a8-8ed0247a2a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:54.446136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1162/2308] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:54.447078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df5dd0-0a3d-4f77-b7a8-8ed0247a2a71 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:54.448326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-42df5dd0-0a3d-4f77-b7a8-8ed0247a2a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:54.475106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1147/2309] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:54 2022] POST /identity/v3/projects => generated 385 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:54.476588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69e297-e262-4748-b62a-1c796c63f936 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.476965 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69e297-e262-4748-b62a-1c796c63f936 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.477305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69e297-e262-4748-b62a-1c796c63f936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.502821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-2c69e297-e262-4748-b62a-1c796c63f936 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:54.504192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2c69e297-e262-4748-b62a-1c796c63f936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:54.519510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df5dd0-0a3d-4f77-b7a8-8ed0247a2a71 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:54.573881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1148/2310] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:54.577808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69e297-e262-4748-b62a-1c796c63f936 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:54.579104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2c69e297-e262-4748-b62a-1c796c63f936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:54.584114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc0c7d-575e-4f51-8c19-c6f0f0c9bd6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:54.584757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc0c7d-575e-4f51-8c19-c6f0f0c9bd6e service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.585038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc0c7d-575e-4f51-8c19-c6f0f0c9bd6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.585310 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc0c7d-575e-4f51-8c19-c6f0f0c9bd6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.590940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69e297-e262-4748-b62a-1c796c63f936 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:54.594752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc0c7d-575e-4f51-8c19-c6f0f0c9bd6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:54.595619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50538c-cd0c-4955-af18-c76681a131c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:54.596227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc0c7d-575e-4f51-8c19-c6f0f0c9bd6e service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:54.596331 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50538c-cd0c-4955-af18-c76681a131c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.596530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50538c-cd0c-4955-af18-c76681a131c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.596803 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50538c-cd0c-4955-af18-c76681a131c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.597431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50538c-cd0c-4955-af18-c76681a131c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:54.599381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50538c-cd0c-4955-af18-c76681a131c5 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:54.608229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1163/2311] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:54.609083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc0c7d-575e-4f51-8c19-c6f0f0c9bd6e service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:54.610975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-62bc0c7d-575e-4f51-8c19-c6f0f0c9bd6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:54.625161 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc0c7d-575e-4f51-8c19-c6f0f0c9bd6e service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:54.631072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:54.631728 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.632012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.632326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] PATH_INFO: `/v3/projects/587064eed23247109ba317e4dda2ef25` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.633477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=587064eed23247109ba317e4dda2ef25)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:54.635367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:54.709043 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1149/2312] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:54 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:54.709870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50538c-cd0c-4955-af18-c76681a131c5 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:54.719607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50538c-cd0c-4955-af18-c76681a131c5 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:54.724781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abde133-da1e-4b9e-a828-736870c77f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:54.725346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abde133-da1e-4b9e-a828-736870c77f88 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.725628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abde133-da1e-4b9e-a828-736870c77f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.725904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abde133-da1e-4b9e-a828-736870c77f88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.726540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abde133-da1e-4b9e-a828-736870c77f88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:54.726841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abde133-da1e-4b9e-a828-736870c77f88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:54.728460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abde133-da1e-4b9e-a828-736870c77f88 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:54.741514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1150/2313] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:28:54.745792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abde133-da1e-4b9e-a828-736870c77f88 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:54.751886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587064eed23247109ba317e4dda2ef25', 'request_id': 'req-b6e83317-65ed-4aa3-9c85-07f1fb826b62'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:54.756199 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abde133-da1e-4b9e-a828-736870c77f88 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:54.759158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587064eed23247109ba317e4dda2ef25', 'request_id': 'req-b6e83317-65ed-4aa3-9c85-07f1fb826b62'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:54.767404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587064eed23247109ba317e4dda2ef25', 'request_id': 'req-b6e83317-65ed-4aa3-9c85-07f1fb826b62'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:54.815535 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb8a07-6acc-4487-a526-084ea6486dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:54.816186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb8a07-6acc-4487-a526-084ea6486dda admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.816500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb8a07-6acc-4487-a526-084ea6486dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.816780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb8a07-6acc-4487-a526-084ea6486dda admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.817433 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb8a07-6acc-4487-a526-084ea6486dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:54.819036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb8a07-6acc-4487-a526-084ea6486dda admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:54.829416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1151/2314] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:54.830200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb8a07-6acc-4487-a526-084ea6486dda admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:54.840267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb8a07-6acc-4487-a526-084ea6486dda admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:54.841624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] The token cache is being invalidate because project 587064eed23247109ba317e4dda2ef25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:28:54.841624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:54.841624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:28:54.843381 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1164/2315] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:28:54 2022] DELETE /identity/v3/projects/587064eed23247109ba317e4dda2ef25 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:54.844132 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:54.845441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:54.845609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782379bd-b4ac-4a5d-b728-e357d81ae1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:54.846320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782379bd-b4ac-4a5d-b728-e357d81ae1cd admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.846809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782379bd-b4ac-4a5d-b728-e357d81ae1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.847087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782379bd-b4ac-4a5d-b728-e357d81ae1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.847724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782379bd-b4ac-4a5d-b728-e357d81ae1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:54.849669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782379bd-b4ac-4a5d-b728-e357d81ae1cd admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:54.858390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e83317-65ed-4aa3-9c85-07f1fb826b62 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:54.863027 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241398-fc34-4285-8ac2-20e4f2b22cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:54.863621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241398-fc34-4285-8ac2-20e4f2b22cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.863904 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241398-fc34-4285-8ac2-20e4f2b22cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.864200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241398-fc34-4285-8ac2-20e4f2b22cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.864870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a241398-fc34-4285-8ac2-20e4f2b22cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:54.866490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a241398-fc34-4285-8ac2-20e4f2b22cd1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:54.880844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1165/2316] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:54.882128 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85761e-f848-439a-8cd8-be92f182927d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.882414 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85761e-f848-439a-8cd8-be92f182927d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.882690 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85761e-f848-439a-8cd8-be92f182927d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.903233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-8b85761e-f848-439a-8cd8-be92f182927d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:54.904835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8b85761e-f848-439a-8cd8-be92f182927d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:54.912337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1166/2317] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:54.913049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85761e-f848-439a-8cd8-be92f182927d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:54.924776 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85761e-f848-439a-8cd8-be92f182927d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:54.929653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83cd3f-8971-48ed-9b9f-30a0089e3a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:54.930329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83cd3f-8971-48ed-9b9f-30a0089e3a28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.930608 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83cd3f-8971-48ed-9b9f-30a0089e3a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.930879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83cd3f-8971-48ed-9b9f-30a0089e3a28 admin admin] PATH_INFO: `/v3/projects/52e1a077737b4eeaa4f463fc0ba1dbf3/users/bf769b105922416c974dea6ad3242e56/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.947337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83cd3f-8971-48ed-9b9f-30a0089e3a28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52e1a077737b4eeaa4f463fc0ba1dbf3, user_id=bf769b105922416c974dea6ad3242e56, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:54.950804 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83cd3f-8971-48ed-9b9f-30a0089e3a28 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:54.953796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1152/2318] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:54 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:54.955253 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a09ef-d49e-447a-870f-238cda95ba47 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:54.955545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a09ef-d49e-447a-870f-238cda95ba47 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:54.955822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a09ef-d49e-447a-870f-238cda95ba47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:54.976803 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-b54a09ef-d49e-447a-870f-238cda95ba47 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:54.978012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b54a09ef-d49e-447a-870f-238cda95ba47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:55.038819 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1153/2319] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:55.039765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a09ef-d49e-447a-870f-238cda95ba47 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:55.041048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b54a09ef-d49e-447a-870f-238cda95ba47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:55.052342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a09ef-d49e-447a-870f-238cda95ba47 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:55.057486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47670b1-0242-43ff-9f77-41383034932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:55.058796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47670b1-0242-43ff-9f77-41383034932b admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.060182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:55.062770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47670b1-0242-43ff-9f77-41383034932b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.062770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47670b1-0242-43ff-9f77-41383034932b admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.062770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47670b1-0242-43ff-9f77-41383034932b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:55.063700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1167/2320] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:54 2022] PUT /identity/v3/projects/52e1a077737b4eeaa4f463fc0ba1dbf3/users/bf769b105922416c974dea6ad3242e56/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:55.064685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83cd3f-8971-48ed-9b9f-30a0089e3a28 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:55.072767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47670b1-0242-43ff-9f77-41383034932b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:55.126228 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83cd3f-8971-48ed-9b9f-30a0089e3a28 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:55.131044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa98fd-fe63-41aa-a2eb-2a1d87d95e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:55.131652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa98fd-fe63-41aa-a2eb-2a1d87d95e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.131937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa98fd-fe63-41aa-a2eb-2a1d87d95e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.132241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa98fd-fe63-41aa-a2eb-2a1d87d95e82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.132940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa98fd-fe63-41aa-a2eb-2a1d87d95e82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:28:55.134926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa98fd-fe63-41aa-a2eb-2a1d87d95e82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:55.134926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa98fd-fe63-41aa-a2eb-2a1d87d95e82 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:55.146996 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1168/2321] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:28:55.148268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033efb3e-32c6-4ed8-a167-6e88d47060c3 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.149058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033efb3e-32c6-4ed8-a167-6e88d47060c3 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.149058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033efb3e-32c6-4ed8-a167-6e88d47060c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.185671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-033efb3e-32c6-4ed8-a167-6e88d47060c3 None None] MFA Rules not processed for user `bf769b105922416c974dea6ad3242e56`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:55.187080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-033efb3e-32c6-4ed8-a167-6e88d47060c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:55.191974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1154/2322] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:55 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:55.193012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47670b1-0242-43ff-9f77-41383034932b admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:55.206228 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47670b1-0242-43ff-9f77-41383034932b admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:55.212467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:55.219639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20996da4-2e03-4695-b002-4568c513c53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:55.220384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20996da4-2e03-4695-b002-4568c513c53b admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.221652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20996da4-2e03-4695-b002-4568c513c53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.222538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20996da4-2e03-4695-b002-4568c513c53b admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.223205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20996da4-2e03-4695-b002-4568c513c53b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:55.225505 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20996da4-2e03-4695-b002-4568c513c53b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:55.263617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1169/2323] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:55.264649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033efb3e-32c6-4ed8-a167-6e88d47060c3 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:55.280272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033efb3e-32c6-4ed8-a167-6e88d47060c3 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:55.285692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77dd87-1305-4fa2-9a69-4775f545aa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:55.286298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77dd87-1305-4fa2-9a69-4775f545aa20 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.286579 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77dd87-1305-4fa2-9a69-4775f545aa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.286853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77dd87-1305-4fa2-9a69-4775f545aa20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.287503 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77dd87-1305-4fa2-9a69-4775f545aa20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:55.289159 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77dd87-1305-4fa2-9a69-4775f545aa20 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:55.306490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1155/2324] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:55 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:55.307379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20996da4-2e03-4695-b002-4568c513c53b admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:55.317643 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1170/2325] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:55.318500 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77dd87-1305-4fa2-9a69-4775f545aa20 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:55.328466 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77dd87-1305-4fa2-9a69-4775f545aa20 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:55.402343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20996da4-2e03-4695-b002-4568c513c53b admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:55.408806 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9511e4-dbbf-4027-8714-8659fb302422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:55.408968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26daa3-9e32-4a7d-b59d-b2ae2b718b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:55.409538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26daa3-9e32-4a7d-b59d-b2ae2b718b40 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.409863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26daa3-9e32-4a7d-b59d-b2ae2b718b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.410462 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26daa3-9e32-4a7d-b59d-b2ae2b718b40 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.410929 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26daa3-9e32-4a7d-b59d-b2ae2b718b40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:55.411367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9511e4-dbbf-4027-8714-8659fb302422 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.412367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9511e4-dbbf-4027-8714-8659fb302422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.412862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9511e4-dbbf-4027-8714-8659fb302422 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.413038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26daa3-9e32-4a7d-b59d-b2ae2b718b40 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:55.415058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9511e4-dbbf-4027-8714-8659fb302422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:55.422341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9511e4-dbbf-4027-8714-8659fb302422 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:55.525225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1171/2326] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:55 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:55.526338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26daa3-9e32-4a7d-b59d-b2ae2b718b40 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:55.539135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26daa3-9e32-4a7d-b59d-b2ae2b718b40 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:55.544927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821b730-92d0-4c3a-9402-e649e8dc1818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:55.545742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821b730-92d0-4c3a-9402-e649e8dc1818 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.546398 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821b730-92d0-4c3a-9402-e649e8dc1818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.546869 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821b730-92d0-4c3a-9402-e649e8dc1818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.547693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821b730-92d0-4c3a-9402-e649e8dc1818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:55.549772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821b730-92d0-4c3a-9402-e649e8dc1818 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:55.575978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1172/2327] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:55.575978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821b730-92d0-4c3a-9402-e649e8dc1818 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:55.576811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1156/2328] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:28:55 2022] POST /identity/v3/projects => generated 383 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:55.577785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9511e4-dbbf-4027-8714-8659fb302422 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:55.591836 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821b730-92d0-4c3a-9402-e649e8dc1818 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:55.684272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9511e4-dbbf-4027-8714-8659fb302422 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:55.696788 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40212b-ae18-4663-917c-e6fc1084bc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:55.697266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40212b-ae18-4663-917c-e6fc1084bc78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.698242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40212b-ae18-4663-917c-e6fc1084bc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.698576 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40212b-ae18-4663-917c-e6fc1084bc78 admin admin] PATH_INFO: `/v3/projects/f9dc90d46bef473da35bdd69f71ae31d/users/703acfc51abf4086bd65e48ec5541905/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.705128 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8f9cf-9c7d-4774-8bef-27f6296a77fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:55.705739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8f9cf-9c7d-4774-8bef-27f6296a77fc admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.706245 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8f9cf-9c7d-4774-8bef-27f6296a77fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.706245 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8f9cf-9c7d-4774-8bef-27f6296a77fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.708122 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8f9cf-9c7d-4774-8bef-27f6296a77fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:55.709556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8f9cf-9c7d-4774-8bef-27f6296a77fc admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:55.733460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40212b-ae18-4663-917c-e6fc1084bc78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9dc90d46bef473da35bdd69f71ae31d, user_id=703acfc51abf4086bd65e48ec5541905, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:55.736858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40212b-ae18-4663-917c-e6fc1084bc78 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:55.744642 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1173/2329] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:55.745762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8f9cf-9c7d-4774-8bef-27f6296a77fc admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:55.760674 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8f9cf-9c7d-4774-8bef-27f6296a77fc admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:55.770910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cca8b5-0c4b-414a-ad52-6f98aae542e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:55.771744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cca8b5-0c4b-414a-ad52-6f98aae542e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.772118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cca8b5-0c4b-414a-ad52-6f98aae542e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.772520 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cca8b5-0c4b-414a-ad52-6f98aae542e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.773259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cca8b5-0c4b-414a-ad52-6f98aae542e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:55.775403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cca8b5-0c4b-414a-ad52-6f98aae542e8 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:55.817203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1157/2330] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:55 2022] PUT /identity/v3/projects/f9dc90d46bef473da35bdd69f71ae31d/users/703acfc51abf4086bd65e48ec5541905/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:55.818040 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40212b-ae18-4663-917c-e6fc1084bc78 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:55.879773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40212b-ae18-4663-917c-e6fc1084bc78 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:55.884690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ff443-4ebb-46b5-8a27-95e8b033606d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:55.885314 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ff443-4ebb-46b5-8a27-95e8b033606d admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.885599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ff443-4ebb-46b5-8a27-95e8b033606d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.885901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ff443-4ebb-46b5-8a27-95e8b033606d admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.886585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ff443-4ebb-46b5-8a27-95e8b033606d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:55.888275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ff443-4ebb-46b5-8a27-95e8b033606d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:55.898626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1158/2331] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:55.899861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c8d40-cf30-44f4-b896-230c9c6589cf None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.900188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c8d40-cf30-44f4-b896-230c9c6589cf None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.900504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c8d40-cf30-44f4-b896-230c9c6589cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.914891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1174/2332] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:28:55 2022] POST /identity/v3/users => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:28:55.917905 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cca8b5-0c4b-414a-ad52-6f98aae542e8 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:55.927595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cca8b5-0c4b-414a-ad52-6f98aae542e8 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:55.930326 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-5d8c8d40-cf30-44f4-b896-230c9c6589cf None None] MFA Rules not processed for user `703acfc51abf4086bd65e48ec5541905`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:55.931538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5d8c8d40-cf30-44f4-b896-230c9c6589cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:55.932300 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa917c8d-b854-4a7f-a5e4-8bc3851d3840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:55.934252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa917c8d-b854-4a7f-a5e4-8bc3851d3840 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:55.934252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa917c8d-b854-4a7f-a5e4-8bc3851d3840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:55.934252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa917c8d-b854-4a7f-a5e4-8bc3851d3840 admin admin] PATH_INFO: `/v3/projects/2e7a16b9d60a483ea46de2778e1b1d54/users/ebd432bffe834f5d8d713243c4d78e21/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:55.957076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa917c8d-b854-4a7f-a5e4-8bc3851d3840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e7a16b9d60a483ea46de2778e1b1d54, user_id=ebd432bffe834f5d8d713243c4d78e21, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:55.959801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa917c8d-b854-4a7f-a5e4-8bc3851d3840 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:56.000589 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1159/2333] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:56.004285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c8d40-cf30-44f4-b896-230c9c6589cf None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:56.022077 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c8d40-cf30-44f4-b896-230c9c6589cf None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:56.027189 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc8f01-2614-4bbe-ac44-fc7a01019b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:56.027870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc8f01-2614-4bbe-ac44-fc7a01019b58 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:56.028151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc8f01-2614-4bbe-ac44-fc7a01019b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:56.028459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc8f01-2614-4bbe-ac44-fc7a01019b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:56.029166 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc8f01-2614-4bbe-ac44-fc7a01019b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:56.030773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc8f01-2614-4bbe-ac44-fc7a01019b58 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:56.043971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1175/2334] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:55 2022] PUT /identity/v3/projects/2e7a16b9d60a483ea46de2778e1b1d54/users/ebd432bffe834f5d8d713243c4d78e21/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:56.045394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa917c8d-b854-4a7f-a5e4-8bc3851d3840 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:56.049955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1160/2335] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:56.051874 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc8f01-2614-4bbe-ac44-fc7a01019b58 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:56.057260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-fa917c8d-b854-4a7f-a5e4-8bc3851d3840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:56.067686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc8f01-2614-4bbe-ac44-fc7a01019b58 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:56.126190 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:56.132390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995dec0-b172-4c81-a893-806825573906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:56.133030 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995dec0-b172-4c81-a893-806825573906 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:56.133199 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995dec0-b172-4c81-a893-806825573906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:56.133477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995dec0-b172-4c81-a893-806825573906 admin admin] PATH_INFO: `/v3/projects/bcafddc65cf143e7bd01a6a4749b2eb0/users/5a0d5af69a204609aaca4977826e1393/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:56.150023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995dec0-b172-4c81-a893-806825573906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcafddc65cf143e7bd01a6a4749b2eb0, user_id=5a0d5af69a204609aaca4977826e1393, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:56.152822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995dec0-b172-4c81-a893-806825573906 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:56.157167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa917c8d-b854-4a7f-a5e4-8bc3851d3840 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:56.161234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530f08c-19f9-480b-b627-0eb1a32112e3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7e768fdd1054e40b6b4910e39a6e533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:56.161911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530f08c-19f9-480b-b627-0eb1a32112e3 service glance] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:56.162267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530f08c-19f9-480b-b627-0eb1a32112e3 service glance] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:56.162614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530f08c-19f9-480b-b627-0eb1a32112e3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:56.221447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530f08c-19f9-480b-b627-0eb1a32112e3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:56.222957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530f08c-19f9-480b-b627-0eb1a32112e3 service glance] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:56.233804 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1176/2336] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:56.235080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e581b-cd9c-4dcb-96a3-8b1141bb3d91 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:56.235371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e581b-cd9c-4dcb-96a3-8b1141bb3d91 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:56.235644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e581b-cd9c-4dcb-96a3-8b1141bb3d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:56.254145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1161/2337] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:56 2022] PUT /identity/v3/projects/bcafddc65cf143e7bd01a6a4749b2eb0/users/5a0d5af69a204609aaca4977826e1393/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:28:56.255584 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995dec0-b172-4c81-a893-806825573906 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:56.269514 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-fe0e581b-cd9c-4dcb-96a3-8b1141bb3d91 None None] MFA Rules not processed for user `ebd432bffe834f5d8d713243c4d78e21`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:56.269886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-fe0e581b-cd9c-4dcb-96a3-8b1141bb3d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:56.343211 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995dec0-b172-4c81-a893-806825573906 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:56.348750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31460ea6-775a-47ee-93fa-2308d015a5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:56.349436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31460ea6-775a-47ee-93fa-2308d015a5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:56.349725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31460ea6-775a-47ee-93fa-2308d015a5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:56.350001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31460ea6-775a-47ee-93fa-2308d015a5c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:56.351043 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31460ea6-775a-47ee-93fa-2308d015a5c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:56.351372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1177/2338] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:56.352892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31460ea6-775a-47ee-93fa-2308d015a5c2 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:56.360938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e581b-cd9c-4dcb-96a3-8b1141bb3d91 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:56.362928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-fe0e581b-cd9c-4dcb-96a3-8b1141bb3d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:56.368541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1162/2339] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:28:56.370915 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31460ea6-775a-47ee-93fa-2308d015a5c2 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:56.381321 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31460ea6-775a-47ee-93fa-2308d015a5c2 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:56.386501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42fe06-dd8d-4180-b626-21a7df2ecf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:56.387449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42fe06-dd8d-4180-b626-21a7df2ecf1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:56.387449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42fe06-dd8d-4180-b626-21a7df2ecf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:56.390442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42fe06-dd8d-4180-b626-21a7df2ecf1c admin admin] PATH_INFO: `/v3/projects/bcafddc65cf143e7bd01a6a4749b2eb0/users/5a0d5af69a204609aaca4977826e1393/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:56.392819 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42fe06-dd8d-4180-b626-21a7df2ecf1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcafddc65cf143e7bd01a6a4749b2eb0, user_id=5a0d5af69a204609aaca4977826e1393, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:56.395764 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42fe06-dd8d-4180-b626-21a7df2ecf1c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:56.408508 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1163/2340] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:28:56 2022] PUT /identity/v3/projects/bcafddc65cf143e7bd01a6a4749b2eb0/users/5a0d5af69a204609aaca4977826e1393/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:28:56.423169 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827d8cc-d80b-414d-8da4-4085e7194b85 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:56.423456 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827d8cc-d80b-414d-8da4-4085e7194b85 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:56.423733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827d8cc-d80b-414d-8da4-4085e7194b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:56.459281 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-b827d8cc-d80b-414d-8da4-4085e7194b85 None None] MFA Rules not processed for user `5a0d5af69a204609aaca4977826e1393`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:28:56.460899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b827d8cc-d80b-414d-8da4-4085e7194b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:56.463666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e581b-cd9c-4dcb-96a3-8b1141bb3d91 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:56.533965 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173297b3-7014-44c4-b1d6-aec483e44b3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:56.535135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173297b3-7014-44c4-b1d6-aec483e44b3b service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:56.535448 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173297b3-7014-44c4-b1d6-aec483e44b3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:56.535734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173297b3-7014-44c4-b1d6-aec483e44b3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:56.590000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1164/2341] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:28:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:28:56.623815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173297b3-7014-44c4-b1d6-aec483e44b3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:56.625354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173297b3-7014-44c4-b1d6-aec483e44b3b service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:56.634495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1178/2342] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:28:56.748936 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:57.132883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:57.259449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827d8cc-d80b-414d-8da4-4085e7194b85 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:57.260662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b827d8cc-d80b-414d-8da4-4085e7194b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:57.274163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827d8cc-d80b-414d-8da4-4085e7194b85 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:57.279044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c14d2-2e8c-4000-a599-08c8d41ba12d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:57.279627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c14d2-2e8c-4000-a599-08c8d41ba12d service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:57.279907 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c14d2-2e8c-4000-a599-08c8d41ba12d service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:57.280216 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c14d2-2e8c-4000-a599-08c8d41ba12d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:57.282136 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ce2c14d2-2e8c-4000-a599-08c8d41ba12d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:28:57.364224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c14d2-2e8c-4000-a599-08c8d41ba12d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:57.365599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c14d2-2e8c-4000-a599-08c8d41ba12d service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:57.376626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1165/2343] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:28:57.434252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:59.384708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:28:59.655510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173297b3-7014-44c4-b1d6-aec483e44b3b service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:28:59.678816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173297b3-7014-44c4-b1d6-aec483e44b3b service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:28:59.678816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dd0d8-fb30-4989-a7ee-11cf33ac7af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:28:59.678816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dd0d8-fb30-4989-a7ee-11cf33ac7af4 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:28:59.678816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dd0d8-fb30-4989-a7ee-11cf33ac7af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:28:59.679313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dd0d8-fb30-4989-a7ee-11cf33ac7af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:28:59.756733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1dd0d8-fb30-4989-a7ee-11cf33ac7af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:28:59.758063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1dd0d8-fb30-4989-a7ee-11cf33ac7af4 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:28:59.768119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1179/2344] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:28:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:29:00.152108 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:01.774138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20effc3-a142-4525-ab7e-bf19ab92a6c1 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:01.775249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20effc3-a142-4525-ab7e-bf19ab92a6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:01.776670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20effc3-a142-4525-ab7e-bf19ab92a6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:01.813254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-e20effc3-a142-4525-ab7e-bf19ab92a6c1 None None] MFA Rules not processed for user `95d668bfaf71417baac3c4336c33ed9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:01.814511 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e20effc3-a142-4525-ab7e-bf19ab92a6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:01.905473 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1166/2345] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:01.909775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dd0d8-fb30-4989-a7ee-11cf33ac7af4 service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:01.911451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-4f1dd0d8-fb30-4989-a7ee-11cf33ac7af4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:01.992983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20effc3-a142-4525-ab7e-bf19ab92a6c1 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:02.005410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20effc3-a142-4525-ab7e-bf19ab92a6c1 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:02.009888 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ae70c-9c03-4622-86a0-5e235fd0f774 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:02.011327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ae70c-9c03-4622-86a0-5e235fd0f774 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:02.011734 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ae70c-9c03-4622-86a0-5e235fd0f774 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:02.012016 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ae70c-9c03-4622-86a0-5e235fd0f774 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:02.022336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ae70c-9c03-4622-86a0-5e235fd0f774 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:02.023780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ae70c-9c03-4622-86a0-5e235fd0f774 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:02.047760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1167/2346] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:29:02.078023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dd0d8-fb30-4989-a7ee-11cf33ac7af4 service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:02.083883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ab515-44da-4ac4-b70e-0e0ca847cfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:02.084611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ab515-44da-4ac4-b70e-0e0ca847cfcb admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:02.084895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ab515-44da-4ac4-b70e-0e0ca847cfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:02.085171 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ab515-44da-4ac4-b70e-0e0ca847cfcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:02.085816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ab515-44da-4ac4-b70e-0e0ca847cfcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:02.087816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ab515-44da-4ac4-b70e-0e0ca847cfcb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:02.261367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:02.268323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1180/2347] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:29:01 2022] POST /identity/v3/projects => generated 371 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:29:02.272767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ae70c-9c03-4622-86a0-5e235fd0f774 service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:02.284235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ae70c-9c03-4622-86a0-5e235fd0f774 service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:02.354319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aea17e-e972-49c9-bcf9-79874e081978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:02.355323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aea17e-e972-49c9-bcf9-79874e081978 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:02.355773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aea17e-e972-49c9-bcf9-79874e081978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:02.356244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aea17e-e972-49c9-bcf9-79874e081978 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:02.357547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aea17e-e972-49c9-bcf9-79874e081978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:02.359694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aea17e-e972-49c9-bcf9-79874e081978 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:02.469976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1168/2348] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:29:02 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:29:02.473998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ab515-44da-4ac4-b70e-0e0ca847cfcb admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:02.485055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ab515-44da-4ac4-b70e-0e0ca847cfcb admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:02.489780 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96d1b7-959c-4832-9413-ceaa26d11b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:02.490517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96d1b7-959c-4832-9413-ceaa26d11b4a admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:02.490871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96d1b7-959c-4832-9413-ceaa26d11b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:02.491325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96d1b7-959c-4832-9413-ceaa26d11b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:02.492123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96d1b7-959c-4832-9413-ceaa26d11b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:02.493775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96d1b7-959c-4832-9413-ceaa26d11b4a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:02.505046 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1181/2349] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:29:02.515908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aea17e-e972-49c9-bcf9-79874e081978 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:02.526329 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aea17e-e972-49c9-bcf9-79874e081978 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:02.531144 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62baf316-1762-4fbc-b1fe-f8cdba26cb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:02.531801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baf316-1762-4fbc-b1fe-f8cdba26cb4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:02.532084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baf316-1762-4fbc-b1fe-f8cdba26cb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:02.532395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baf316-1762-4fbc-b1fe-f8cdba26cb4c admin admin] PATH_INFO: `/v3/projects/32871c359a34494cbd142bde9abfaa55/users/cee743990ef84e1594242c1c51bcc13e/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:02.552414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62baf316-1762-4fbc-b1fe-f8cdba26cb4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=32871c359a34494cbd142bde9abfaa55, user_id=cee743990ef84e1594242c1c51bcc13e, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:02.555386 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62baf316-1762-4fbc-b1fe-f8cdba26cb4c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:02.594920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:02.675676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1169/2350] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:29:02 2022] PUT /identity/v3/projects/32871c359a34494cbd142bde9abfaa55/users/cee743990ef84e1594242c1c51bcc13e/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:02.679720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96d1b7-959c-4832-9413-ceaa26d11b4a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:02.691392 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96d1b7-959c-4832-9413-ceaa26d11b4a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:02.778775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e6115-1fda-4313-ad44-105d15c9a397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:02.779386 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e6115-1fda-4313-ad44-105d15c9a397 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:02.779971 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e6115-1fda-4313-ad44-105d15c9a397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:02.780279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e6115-1fda-4313-ad44-105d15c9a397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:02.781340 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e6115-1fda-4313-ad44-105d15c9a397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:02.783309 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e6115-1fda-4313-ad44-105d15c9a397 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:02.795753 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1182/2351] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:29:02.799826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62baf316-1762-4fbc-b1fe-f8cdba26cb4c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:02.818734 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62baf316-1762-4fbc-b1fe-f8cdba26cb4c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:02.824013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1497663-facd-43f5-8ab6-4683c33d24f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:02.824920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1497663-facd-43f5-8ab6-4683c33d24f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:02.825566 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1497663-facd-43f5-8ab6-4683c33d24f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:02.825863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1497663-facd-43f5-8ab6-4683c33d24f9 admin admin] PATH_INFO: `/v3/projects/32871c359a34494cbd142bde9abfaa55/users/cee743990ef84e1594242c1c51bcc13e/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:02.827812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1497663-facd-43f5-8ab6-4683c33d24f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32871c359a34494cbd142bde9abfaa55, user_id=cee743990ef84e1594242c1c51bcc13e, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:02.830998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1497663-facd-43f5-8ab6-4683c33d24f9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:02.846555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1170/2352] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:29:02 2022] PUT /identity/v3/projects/32871c359a34494cbd142bde9abfaa55/users/cee743990ef84e1594242c1c51bcc13e/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:03.783244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:04.097617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:06.812515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:07.065709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:09.292858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:09.579173 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:09.603665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d39e-02cf-414f-b5cc-3432c9ad79d1 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:09.605014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d39e-02cf-414f-b5cc-3432c9ad79d1 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:09.605594 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d39e-02cf-414f-b5cc-3432c9ad79d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:09.665654 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-97a0d39e-02cf-414f-b5cc-3432c9ad79d1 None None] MFA Rules not processed for user `cee743990ef84e1594242c1c51bcc13e`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:09.666993 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-97a0d39e-02cf-414f-b5cc-3432c9ad79d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:09.743736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1497663-facd-43f5-8ab6-4683c33d24f9 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:09.769357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1183/2353] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:09.783632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0d39e-02cf-414f-b5cc-3432c9ad79d1 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:09.786225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-97a0d39e-02cf-414f-b5cc-3432c9ad79d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:09.838286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1497663-facd-43f5-8ab6-4683c33d24f9 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:09.847064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71f193-109b-4f69-a111-21ae53a98545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:09.855603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71f193-109b-4f69-a111-21ae53a98545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:09.855603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71f193-109b-4f69-a111-21ae53a98545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:09.855603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71f193-109b-4f69-a111-21ae53a98545 admin admin] PATH_INFO: `/v3/users/2eb79017317e43189f662d4fe8d3f16e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:09.856733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71f193-109b-4f69-a111-21ae53a98545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb79017317e43189f662d4fe8d3f16e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:09.859185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71f193-109b-4f69-a111-21ae53a98545 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:09.923657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0d39e-02cf-414f-b5cc-3432c9ad79d1 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:09.931681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30d022-e345-44cb-a025-d7cbac3d6c17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:09.932935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30d022-e345-44cb-a025-d7cbac3d6c17 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:09.933230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30d022-e345-44cb-a025-d7cbac3d6c17 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:09.933507 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30d022-e345-44cb-a025-d7cbac3d6c17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:10.005923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30d022-e345-44cb-a025-d7cbac3d6c17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:10.007702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30d022-e345-44cb-a025-d7cbac3d6c17 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:10.017669 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1184/2354] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:29:10.018523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30d022-e345-44cb-a025-d7cbac3d6c17 service nova] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:10.031141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30d022-e345-44cb-a025-d7cbac3d6c17 service nova] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:10.040127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97ef77-ed5d-4c83-9073-4fed20d72195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:10.041487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97ef77-ed5d-4c83-9073-4fed20d72195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:10.041794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97ef77-ed5d-4c83-9073-4fed20d72195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:10.042204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97ef77-ed5d-4c83-9073-4fed20d72195 admin admin] PATH_INFO: `/v3/users/ebd432bffe834f5d8d713243c4d78e21` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:10.043586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97ef77-ed5d-4c83-9073-4fed20d72195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd432bffe834f5d8d713243c4d78e21)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:10.046031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97ef77-ed5d-4c83-9073-4fed20d72195 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:10.106272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ea71f193-109b-4f69-a111-21ae53a98545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb79017317e43189f662d4fe8d3f16e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.107220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ea71f193-109b-4f69-a111-21ae53a98545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb79017317e43189f662d4fe8d3f16e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.115596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ea71f193-109b-4f69-a111-21ae53a98545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb79017317e43189f662d4fe8d3f16e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.188277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ea71f193-109b-4f69-a111-21ae53a98545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb79017317e43189f662d4fe8d3f16e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.188900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ea71f193-109b-4f69-a111-21ae53a98545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb79017317e43189f662d4fe8d3f16e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.200944 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:10.208549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1171/2355] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:29:09 2022] DELETE /identity/v3/users/2eb79017317e43189f662d4fe8d3f16e => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:10.212967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57823f-8be3-41e1-9d82-81a8e58eb73f None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:10.213257 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57823f-8be3-41e1-9d82-81a8e58eb73f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:10.213563 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57823f-8be3-41e1-9d82-81a8e58eb73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:10.232833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-da57823f-8be3-41e1-9d82-81a8e58eb73f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:10.234037 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-da57823f-8be3-41e1-9d82-81a8e58eb73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:10.264375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9c97ef77-ed5d-4c83-9073-4fed20d72195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd432bffe834f5d8d713243c4d78e21'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.271721 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9c97ef77-ed5d-4c83-9073-4fed20d72195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd432bffe834f5d8d713243c4d78e21'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.294235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1172/2356] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:29:10.303987 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57823f-8be3-41e1-9d82-81a8e58eb73f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:10.305828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-da57823f-8be3-41e1-9d82-81a8e58eb73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:10.368828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9c97ef77-ed5d-4c83-9073-4fed20d72195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd432bffe834f5d8d713243c4d78e21'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.369612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9c97ef77-ed5d-4c83-9073-4fed20d72195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd432bffe834f5d8d713243c4d78e21'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.381228 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57823f-8be3-41e1-9d82-81a8e58eb73f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:10.388200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39d248-0a19-493b-8e33-0ec881ebdd36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:10.390705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39d248-0a19-493b-8e33-0ec881ebdd36 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:10.391014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39d248-0a19-493b-8e33-0ec881ebdd36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:10.391283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39d248-0a19-493b-8e33-0ec881ebdd36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:10.393459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4f39d248-0a19-493b-8e33-0ec881ebdd36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:10.399369 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-9c97ef77-ed5d-4c83-9073-4fed20d72195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd432bffe834f5d8d713243c4d78e21'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.405351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1185/2357] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:29:10 2022] DELETE /identity/v3/users/ebd432bffe834f5d8d713243c4d78e21 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:10.406927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39d248-0a19-493b-8e33-0ec881ebdd36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:10.408627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39d248-0a19-493b-8e33-0ec881ebdd36 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:10.411314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff3c1a-84e0-47fd-b117-a9ef163af55e None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:10.411637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff3c1a-84e0-47fd-b117-a9ef163af55e None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:10.411915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff3c1a-84e0-47fd-b117-a9ef163af55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:10.419066 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1173/2358] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:29:10.431498 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-bcff3c1a-84e0-47fd-b117-a9ef163af55e None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:10.433308 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bcff3c1a-84e0-47fd-b117-a9ef163af55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:10.507383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1186/2359] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:29:10.518225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39d248-0a19-493b-8e33-0ec881ebdd36 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:10.519931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4f39d248-0a19-493b-8e33-0ec881ebdd36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:10.531630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39d248-0a19-493b-8e33-0ec881ebdd36 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:10.581119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da139706-a63a-4557-8b0d-49f115038f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:10.581743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da139706-a63a-4557-8b0d-49f115038f55 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:10.582022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da139706-a63a-4557-8b0d-49f115038f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:10.588346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da139706-a63a-4557-8b0d-49f115038f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:10.591851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da139706-a63a-4557-8b0d-49f115038f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:10.593328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da139706-a63a-4557-8b0d-49f115038f55 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:10.602570 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1174/2360] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:29:11.044853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff3c1a-84e0-47fd-b117-a9ef163af55e None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:11.044853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bcff3c1a-84e0-47fd-b117-a9ef163af55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:11.052490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff3c1a-84e0-47fd-b117-a9ef163af55e None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:11.063499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:11.064471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:11.064780 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:11.065212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] PATH_INFO: `/v3/projects/4d852297e12a4b1c942d6661b3edd97a` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:11.067091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d852297e12a4b1c942d6661b3edd97a)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:11.069108 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:11.196347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da139706-a63a-4557-8b0d-49f115038f55 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:11.197850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-da139706-a63a-4557-8b0d-49f115038f55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:11.227205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da139706-a63a-4557-8b0d-49f115038f55 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:11.237781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94088455-06d1-48ca-8cbe-4e255c05c44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:11.238927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94088455-06d1-48ca-8cbe-4e255c05c44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:11.239211 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94088455-06d1-48ca-8cbe-4e255c05c44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:11.239489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94088455-06d1-48ca-8cbe-4e255c05c44e admin admin] PATH_INFO: `/v3/projects/2e7a16b9d60a483ea46de2778e1b1d54` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:11.244428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94088455-06d1-48ca-8cbe-4e255c05c44e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e7a16b9d60a483ea46de2778e1b1d54)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:11.248906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94088455-06d1-48ca-8cbe-4e255c05c44e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:11.271964 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d852297e12a4b1c942d6661b3edd97a', 'request_id': 'req-d5ca93a8-0010-495b-9caf-f1cd555e288d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.280605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d852297e12a4b1c942d6661b3edd97a', 'request_id': 'req-d5ca93a8-0010-495b-9caf-f1cd555e288d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.287594 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d852297e12a4b1c942d6661b3edd97a', 'request_id': 'req-d5ca93a8-0010-495b-9caf-f1cd555e288d'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.375883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] The token cache is being invalidate because project 4d852297e12a4b1c942d6661b3edd97a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:29:11.378552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.380209 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.387743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1187/2361] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:29:11 2022] DELETE /identity/v3/projects/4d852297e12a4b1c942d6661b3edd97a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:11.388946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:11.390439 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:11.411322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94088455-06d1-48ca-8cbe-4e255c05c44e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e7a16b9d60a483ea46de2778e1b1d54', 'request_id': 'req-94088455-06d1-48ca-8cbe-4e255c05c44e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.417801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94088455-06d1-48ca-8cbe-4e255c05c44e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e7a16b9d60a483ea46de2778e1b1d54', 'request_id': 'req-94088455-06d1-48ca-8cbe-4e255c05c44e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.469422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca93a8-0010-495b-9caf-f1cd555e288d admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:11.474938 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94088455-06d1-48ca-8cbe-4e255c05c44e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e7a16b9d60a483ea46de2778e1b1d54', 'request_id': 'req-94088455-06d1-48ca-8cbe-4e255c05c44e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.490995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94088455-06d1-48ca-8cbe-4e255c05c44e admin admin] The token cache is being invalidate because project 2e7a16b9d60a483ea46de2778e1b1d54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:29:11.490995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94088455-06d1-48ca-8cbe-4e255c05c44e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.491717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-94088455-06d1-48ca-8cbe-4e255c05c44e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.493951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1175/2362] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:29:11 2022] DELETE /identity/v3/projects/2e7a16b9d60a483ea46de2778e1b1d54 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:29:11.502928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257659f-c656-4e56-adb0-8263c21ccd6c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:11.502928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257659f-c656-4e56-adb0-8263c21ccd6c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:11.502928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257659f-c656-4e56-adb0-8263c21ccd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:11.522104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f257659f-c656-4e56-adb0-8263c21ccd6c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:11.524826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f257659f-c656-4e56-adb0-8263c21ccd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:11.530504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:11.531138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:11.531454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:11.531727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] PATH_INFO: `/v3/users/bf769b105922416c974dea6ad3242e56` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:11.532877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf769b105922416c974dea6ad3242e56)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:11.534783 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:11.578557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:11.589948 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1176/2363] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:11.597599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14364b0-99cf-4d5d-a4d3-417d35010b92 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:11.597874 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14364b0-99cf-4d5d-a4d3-417d35010b92 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:11.598147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14364b0-99cf-4d5d-a4d3-417d35010b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:11.619675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-d14364b0-99cf-4d5d-a4d3-417d35010b92 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:11.621116 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-d14364b0-99cf-4d5d-a4d3-417d35010b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:11.629350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1177/2364] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:29:11.633574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14364b0-99cf-4d5d-a4d3-417d35010b92 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:11.646618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14364b0-99cf-4d5d-a4d3-417d35010b92 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:11.654140 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0ec73-92e7-4709-be6d-d0145295b321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:11.657936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0ec73-92e7-4709-be6d-d0145295b321 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:11.657936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0ec73-92e7-4709-be6d-d0145295b321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:11.657936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0ec73-92e7-4709-be6d-d0145295b321 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:11.657936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0ec73-92e7-4709-be6d-d0145295b321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:29:11.657936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0ec73-92e7-4709-be6d-d0145295b321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:11.658646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0ec73-92e7-4709-be6d-d0145295b321 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:11.671970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1178/2365] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:29:11.675978 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0ec73-92e7-4709-be6d-d0145295b321 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:11.686635 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0ec73-92e7-4709-be6d-d0145295b321 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:11.691729 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb0f77-568b-4d2e-95d9-d6196f4e08b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:11.692324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb0f77-568b-4d2e-95d9-d6196f4e08b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:11.692605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb0f77-568b-4d2e-95d9-d6196f4e08b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:11.693002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb0f77-568b-4d2e-95d9-d6196f4e08b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:11.693667 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb0f77-568b-4d2e-95d9-d6196f4e08b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:11.695677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb0f77-568b-4d2e-95d9-d6196f4e08b7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:11.742654 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf769b105922416c974dea6ad3242e56'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.750016 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf769b105922416c974dea6ad3242e56'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.825928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1179/2366] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:29:11 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:29:11.830193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb0f77-568b-4d2e-95d9-d6196f4e08b7 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:11.851912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf769b105922416c974dea6ad3242e56'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.852957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf769b105922416c974dea6ad3242e56'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.874841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf769b105922416c974dea6ad3242e56'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:11.885260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1188/2367] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:29:11 2022] DELETE /identity/v3/users/bf769b105922416c974dea6ad3242e56 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:11.910919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb0f77-568b-4d2e-95d9-d6196f4e08b7 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:11.922374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:11.989408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef48744-22a1-4f4c-a8a2-8f6f4d30b6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:11.990151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef48744-22a1-4f4c-a8a2-8f6f4d30b6ed admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:11.990553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef48744-22a1-4f4c-a8a2-8f6f4d30b6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:11.990906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef48744-22a1-4f4c-a8a2-8f6f4d30b6ed admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:11.991660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef48744-22a1-4f4c-a8a2-8f6f4d30b6ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:11.994132 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef48744-22a1-4f4c-a8a2-8f6f4d30b6ed admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:12.109019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1180/2368] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:29:11 2022] POST /identity/v3/users => generated 289 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:29:12.113876 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:12.116603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:12.140837 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cba95-23b5-4978-a731-1cf93da1bc63 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:12.146276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98bcd1b-86dc-407b-a23e-c15a6ed591e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:12.146973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bcd1b-86dc-407b-a23e-c15a6ed591e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:12.147487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bcd1b-86dc-407b-a23e-c15a6ed591e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:12.147840 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bcd1b-86dc-407b-a23e-c15a6ed591e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:12.148591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98bcd1b-86dc-407b-a23e-c15a6ed591e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:12.150275 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98bcd1b-86dc-407b-a23e-c15a6ed591e1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:12.163041 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1189/2369] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:29:12.166926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef48744-22a1-4f4c-a8a2-8f6f4d30b6ed admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:12.190924 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef48744-22a1-4f4c-a8a2-8f6f4d30b6ed admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:12.196887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800374fe-f500-4c58-adff-014da9a35b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:12.197638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800374fe-f500-4c58-adff-014da9a35b98 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:12.198056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800374fe-f500-4c58-adff-014da9a35b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:12.198913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800374fe-f500-4c58-adff-014da9a35b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:12.199174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800374fe-f500-4c58-adff-014da9a35b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:12.201057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800374fe-f500-4c58-adff-014da9a35b98 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:12.212777 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1181/2370] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:29:12.217431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98bcd1b-86dc-407b-a23e-c15a6ed591e1 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:12.228098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98bcd1b-86dc-407b-a23e-c15a6ed591e1 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:12.234008 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e04b3d-3d3d-49e6-bedc-6ae5e5417baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:12.234694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e04b3d-3d3d-49e6-bedc-6ae5e5417baf admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:12.234973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e04b3d-3d3d-49e6-bedc-6ae5e5417baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:12.235432 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e04b3d-3d3d-49e6-bedc-6ae5e5417baf admin admin] PATH_INFO: `/v3/projects/bb27b637f41448b6b18549867edb2319/users/15150cfeaec8407187afddfd3d752fd9/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:12.254336 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e04b3d-3d3d-49e6-bedc-6ae5e5417baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb27b637f41448b6b18549867edb2319, user_id=15150cfeaec8407187afddfd3d752fd9, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:12.257349 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e04b3d-3d3d-49e6-bedc-6ae5e5417baf admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:12.372203 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1190/2371] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:29:12 2022] PUT /identity/v3/projects/bb27b637f41448b6b18549867edb2319/users/15150cfeaec8407187afddfd3d752fd9/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:12.383641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e927aa-dda6-4a8a-82f0-804bfc3b7269 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:12.383998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e927aa-dda6-4a8a-82f0-804bfc3b7269 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:12.384374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e927aa-dda6-4a8a-82f0-804bfc3b7269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:12.389770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e04b3d-3d3d-49e6-bedc-6ae5e5417baf admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:12.391519 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-99e04b3d-3d3d-49e6-bedc-6ae5e5417baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:12.430013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f9e927aa-dda6-4a8a-82f0-804bfc3b7269 None None] MFA Rules not processed for user `15150cfeaec8407187afddfd3d752fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:12.431251 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f9e927aa-dda6-4a8a-82f0-804bfc3b7269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:12.481274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e04b3d-3d3d-49e6-bedc-6ae5e5417baf admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:12.486947 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:12.487595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:12.488406 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:12.488595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] PATH_INFO: `/v3/projects/52e1a077737b4eeaa4f463fc0ba1dbf3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:12.490019 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] RBAC: Authorizing `identity:delete_project(project_id=52e1a077737b4eeaa4f463fc0ba1dbf3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:12.492066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:12.550105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1182/2372] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:29:12.554691 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e927aa-dda6-4a8a-82f0-804bfc3b7269 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:12.567837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e927aa-dda6-4a8a-82f0-804bfc3b7269 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:12.572473 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ec975-1af7-47a1-ada2-5701bdad6342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:12.573142 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ec975-1af7-47a1-ada2-5701bdad6342 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:12.573426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ec975-1af7-47a1-ada2-5701bdad6342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:12.573700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ec975-1af7-47a1-ada2-5701bdad6342 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:12.574335 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ec975-1af7-47a1-ada2-5701bdad6342 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:12.576300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ec975-1af7-47a1-ada2-5701bdad6342 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:12.681170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1183/2373] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:29:12 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:29:12.688792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ec975-1af7-47a1-ada2-5701bdad6342 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:12.699571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52e1a077737b4eeaa4f463fc0ba1dbf3', 'request_id': 'req-b3aaa5f0-dc11-40d0-b803-971f7093443e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:12.707141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52e1a077737b4eeaa4f463fc0ba1dbf3', 'request_id': 'req-b3aaa5f0-dc11-40d0-b803-971f7093443e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:12.714872 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52e1a077737b4eeaa4f463fc0ba1dbf3', 'request_id': 'req-b3aaa5f0-dc11-40d0-b803-971f7093443e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:12.762451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] The token cache is being invalidate because project 52e1a077737b4eeaa4f463fc0ba1dbf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:29:12.763196 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:12.768361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:12.770034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1191/2374] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:29:12 2022] DELETE /identity/v3/projects/52e1a077737b4eeaa4f463fc0ba1dbf3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:12.771018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:12.781635 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:12.781816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ec975-1af7-47a1-ada2-5701bdad6342 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:12.781816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad24913-239c-4ade-8ba4-28fe9e1b4e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:12.781816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad24913-239c-4ade-8ba4-28fe9e1b4e78 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:12.781816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad24913-239c-4ade-8ba4-28fe9e1b4e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:12.782242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad24913-239c-4ade-8ba4-28fe9e1b4e78 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:12.782329 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad24913-239c-4ade-8ba4-28fe9e1b4e78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:12.784283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad24913-239c-4ade-8ba4-28fe9e1b4e78 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:12.864515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1184/2375] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:29:12 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:29:12.868310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad24913-239c-4ade-8ba4-28fe9e1b4e78 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:12.870621 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5ad24913-239c-4ade-8ba4-28fe9e1b4e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:12.890045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aaa5f0-dc11-40d0-b803-971f7093443e admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:12.892237 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad24913-239c-4ade-8ba4-28fe9e1b4e78 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:12.896450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df406108-9d28-4d86-afa1-4e1785058f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:12.897137 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df406108-9d28-4d86-afa1-4e1785058f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:12.897464 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df406108-9d28-4d86-afa1-4e1785058f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:12.897897 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df406108-9d28-4d86-afa1-4e1785058f93 admin admin] PATH_INFO: `/v3/users/5a0d5af69a204609aaca4977826e1393` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:12.899265 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df406108-9d28-4d86-afa1-4e1785058f93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a0d5af69a204609aaca4977826e1393)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:12.901326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df406108-9d28-4d86-afa1-4e1785058f93 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:12.905740 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4558c-d3bb-44d5-ae97-3f1aab70bf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:12.906438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4558c-d3bb-44d5-ae97-3f1aab70bf7f admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:12.906824 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4558c-d3bb-44d5-ae97-3f1aab70bf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:12.907219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4558c-d3bb-44d5-ae97-3f1aab70bf7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:12.908031 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4558c-d3bb-44d5-ae97-3f1aab70bf7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:12.909774 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4558c-d3bb-44d5-ae97-3f1aab70bf7f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:12.922134 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1185/2376] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:29:12.926225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4558c-d3bb-44d5-ae97-3f1aab70bf7f admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:12.937820 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4558c-d3bb-44d5-ae97-3f1aab70bf7f admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:12.950338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61b2e2-5c65-4a2b-93b4-b615c14c643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:12.951421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61b2e2-5c65-4a2b-93b4-b615c14c643f admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:12.951864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61b2e2-5c65-4a2b-93b4-b615c14c643f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:12.952361 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61b2e2-5c65-4a2b-93b4-b615c14c643f admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:12.953339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61b2e2-5c65-4a2b-93b4-b615c14c643f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:12.957911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61b2e2-5c65-4a2b-93b4-b615c14c643f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:12.971848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1186/2377] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:29:12.976145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61b2e2-5c65-4a2b-93b4-b615c14c643f admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:12.987586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61b2e2-5c65-4a2b-93b4-b615c14c643f admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:12.993350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f58077-68f5-467b-88d0-88e03e50cdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:12.994010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f58077-68f5-467b-88d0-88e03e50cdc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:12.994297 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f58077-68f5-467b-88d0-88e03e50cdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:12.994575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f58077-68f5-467b-88d0-88e03e50cdc5 admin admin] PATH_INFO: `/v3/projects/db3764fd345b476fa94f5f68d1084e15/users/1ce50a8460864cd4b5fa5fb0d901d166/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:13.013145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f58077-68f5-467b-88d0-88e03e50cdc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3764fd345b476fa94f5f68d1084e15, user_id=1ce50a8460864cd4b5fa5fb0d901d166, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:13.015936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f58077-68f5-467b-88d0-88e03e50cdc5 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:13.110807 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1187/2378] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:29:12 2022] PUT /identity/v3/projects/db3764fd345b476fa94f5f68d1084e15/users/1ce50a8460864cd4b5fa5fb0d901d166/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:29:13.121776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb59235-1d20-47a1-b052-03a3da4dd102 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:13.122063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb59235-1d20-47a1-b052-03a3da4dd102 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:13.122344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb59235-1d20-47a1-b052-03a3da4dd102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:13.151904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-acb59235-1d20-47a1-b052-03a3da4dd102 None None] MFA Rules not processed for user `1ce50a8460864cd4b5fa5fb0d901d166`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:13.153146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-acb59235-1d20-47a1-b052-03a3da4dd102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:13.239326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-df406108-9d28-4d86-afa1-4e1785058f93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a0d5af69a204609aaca4977826e1393'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:13.253877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-df406108-9d28-4d86-afa1-4e1785058f93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a0d5af69a204609aaca4977826e1393'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:13.263132 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1188/2379] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:29:13.268110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb59235-1d20-47a1-b052-03a3da4dd102 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:13.339410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb59235-1d20-47a1-b052-03a3da4dd102 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:13.344388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4160e-4ea8-4d1f-9c87-1a05761542f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:13.345113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4160e-4ea8-4d1f-9c87-1a05761542f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:13.345466 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4160e-4ea8-4d1f-9c87-1a05761542f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:13.345810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4160e-4ea8-4d1f-9c87-1a05761542f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:13.346552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4160e-4ea8-4d1f-9c87-1a05761542f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:13.349134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-df406108-9d28-4d86-afa1-4e1785058f93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a0d5af69a204609aaca4977826e1393'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:13.349888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-df406108-9d28-4d86-afa1-4e1785058f93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a0d5af69a204609aaca4977826e1393'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:13.350197 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4160e-4ea8-4d1f-9c87-1a05761542f9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:13.368704 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-df406108-9d28-4d86-afa1-4e1785058f93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a0d5af69a204609aaca4977826e1393'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:13.375850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1192/2380] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:29:12 2022] DELETE /identity/v3/users/5a0d5af69a204609aaca4977826e1393 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:13.377227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabd2b6-6371-4874-abd1-3dff664fb368 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:13.377516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabd2b6-6371-4874-abd1-3dff664fb368 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:13.377786 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabd2b6-6371-4874-abd1-3dff664fb368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:13.397692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ddabd2b6-6371-4874-abd1-3dff664fb368 None None] MFA Rules not processed for user `95d668bfaf71417baac3c4336c33ed9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:13.398931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ddabd2b6-6371-4874-abd1-3dff664fb368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:13.448500 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1189/2381] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:29:13 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:29:13.452099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4160e-4ea8-4d1f-9c87-1a05761542f9 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:13.458200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-69a4160e-4ea8-4d1f-9c87-1a05761542f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:13.471998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1193/2382] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:13.560879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4160e-4ea8-4d1f-9c87-1a05761542f9 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:13.575170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e2d0b-6867-4968-80b3-42cf373037a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:13.580406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e2d0b-6867-4968-80b3-42cf373037a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:13.580406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e2d0b-6867-4968-80b3-42cf373037a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:13.580406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e2d0b-6867-4968-80b3-42cf373037a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:13.580406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e2d0b-6867-4968-80b3-42cf373037a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:13.580406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e2d0b-6867-4968-80b3-42cf373037a5 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:13.765476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1190/2383] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:29:13 2022] POST /identity/v3/users => generated 295 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:29:13.769547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabd2b6-6371-4874-abd1-3dff664fb368 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:13.922297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabd2b6-6371-4874-abd1-3dff664fb368 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:13.928997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e83efd-5f0d-4853-af91-299fcf4319e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:13.929624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e83efd-5f0d-4853-af91-299fcf4319e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:13.929951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e83efd-5f0d-4853-af91-299fcf4319e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:13.930231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e83efd-5f0d-4853-af91-299fcf4319e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:13.931023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e83efd-5f0d-4853-af91-299fcf4319e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:13.933258 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e83efd-5f0d-4853-af91-299fcf4319e4 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:13.947472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1194/2384] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:29:13.951486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e2d0b-6867-4968-80b3-42cf373037a5 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:13.964789 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e2d0b-6867-4968-80b3-42cf373037a5 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:13.969617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80008a3-b3a1-47ab-bfd9-0c4e9d6f28f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:13.970279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80008a3-b3a1-47ab-bfd9-0c4e9d6f28f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:13.970567 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80008a3-b3a1-47ab-bfd9-0c4e9d6f28f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:13.970850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80008a3-b3a1-47ab-bfd9-0c4e9d6f28f3 admin admin] PATH_INFO: `/v3/projects/7ba4691d0f3f4a3f9dcf72a8b385a829/users/d363e1a47cf247a0a7c4aeeba9f408ac/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:13.986792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80008a3-b3a1-47ab-bfd9-0c4e9d6f28f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba4691d0f3f4a3f9dcf72a8b385a829, user_id=d363e1a47cf247a0a7c4aeeba9f408ac, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:13.989496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80008a3-b3a1-47ab-bfd9-0c4e9d6f28f3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:14.186302 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e83efd-5f0d-4853-af91-299fcf4319e4 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:14.188418 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d7e83efd-5f0d-4853-af91-299fcf4319e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:14.202911 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e83efd-5f0d-4853-af91-299fcf4319e4 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:14.222007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1191/2385] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:29:13 2022] PUT /identity/v3/projects/7ba4691d0f3f4a3f9dcf72a8b385a829/users/d363e1a47cf247a0a7c4aeeba9f408ac/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:29:14.222251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:14.223717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:14.225299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80008a3-b3a1-47ab-bfd9-0c4e9d6f28f3 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:14.225550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:14.226677 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3 admin admin] PATH_INFO: `/v3/projects/bcafddc65cf143e7bd01a6a4749b2eb0` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:14.228219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcafddc65cf143e7bd01a6a4749b2eb0)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:14.231271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:14.290051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80008a3-b3a1-47ab-bfd9-0c4e9d6f28f3 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:14.295833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345c413-fdf6-4c4a-890a-94ac8d0a3d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:14.296544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345c413-fdf6-4c4a-890a-94ac8d0a3d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:14.297014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345c413-fdf6-4c4a-890a-94ac8d0a3d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:14.301512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345c413-fdf6-4c4a-890a-94ac8d0a3d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:14.301512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345c413-fdf6-4c4a-890a-94ac8d0a3d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:14.301512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345c413-fdf6-4c4a-890a-94ac8d0a3d08 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:14.313862 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1192/2386] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:29:14.318311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345c413-fdf6-4c4a-890a-94ac8d0a3d08 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:14.329167 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345c413-fdf6-4c4a-890a-94ac8d0a3d08 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:14.334138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6a6eb-0287-4eb2-9026-b4c8ac286e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:14.335058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6a6eb-0287-4eb2-9026-b4c8ac286e34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:14.335346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6a6eb-0287-4eb2-9026-b4c8ac286e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:14.339301 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6a6eb-0287-4eb2-9026-b4c8ac286e34 admin admin] PATH_INFO: `/v3/projects/7ba4691d0f3f4a3f9dcf72a8b385a829/users/d363e1a47cf247a0a7c4aeeba9f408ac/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:14.339301 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6a6eb-0287-4eb2-9026-b4c8ac286e34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba4691d0f3f4a3f9dcf72a8b385a829, user_id=d363e1a47cf247a0a7c4aeeba9f408ac, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:14.341162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6a6eb-0287-4eb2-9026-b4c8ac286e34 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:14.363607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1193/2387] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:29:14 2022] PUT /identity/v3/projects/7ba4691d0f3f4a3f9dcf72a8b385a829/users/d363e1a47cf247a0a7c4aeeba9f408ac/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:14.373670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4ff34-8d83-4bdc-8592-a75622921d55 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:14.373957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4ff34-8d83-4bdc-8592-a75622921d55 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:14.374232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4ff34-8d83-4bdc-8592-a75622921d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:14.393618 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcafddc65cf143e7bd01a6a4749b2eb0', 'request_id': 'req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:14.403716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcafddc65cf143e7bd01a6a4749b2eb0', 'request_id': 'req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:14.413935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcafddc65cf143e7bd01a6a4749b2eb0', 'request_id': 'req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:14.416634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7ba4ff34-8d83-4bdc-8592-a75622921d55 None None] MFA Rules not processed for user `d363e1a47cf247a0a7c4aeeba9f408ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:14.418830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7ba4ff34-8d83-4bdc-8592-a75622921d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:14.496755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:14.530472 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1194/2388] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:29:14.545553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4ff34-8d83-4bdc-8592-a75622921d55 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:14.545734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3 admin admin] The token cache is being invalidate because project bcafddc65cf143e7bd01a6a4749b2eb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:29:14.545897 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:14.546609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8b246fd0-bc3b-431c-a551-cb2f7185a7d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:14.547472 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7ba4ff34-8d83-4bdc-8592-a75622921d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:14.551401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1195/2389] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:29:14 2022] DELETE /identity/v3/projects/bcafddc65cf143e7bd01a6a4749b2eb0 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:14.556269 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c4f00-ee76-434a-8528-86fb268eff1d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:14.556890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c4f00-ee76-434a-8528-86fb268eff1d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:14.557165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c4f00-ee76-434a-8528-86fb268eff1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:14.591848 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-c42c4f00-ee76-434a-8528-86fb268eff1d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:14.593366 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c42c4f00-ee76-434a-8528-86fb268eff1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:14.617940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4ff34-8d83-4bdc-8592-a75622921d55 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:14.624473 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d10ece-7c9e-4476-8c83-0dc286a52ac9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:14.625282 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d10ece-7c9e-4476-8c83-0dc286a52ac9 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:14.625562 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d10ece-7c9e-4476-8c83-0dc286a52ac9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:14.626175 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d10ece-7c9e-4476-8c83-0dc286a52ac9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:14.628005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c2d10ece-7c9e-4476-8c83-0dc286a52ac9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:14.735825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d10ece-7c9e-4476-8c83-0dc286a52ac9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:14.735825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d10ece-7c9e-4476-8c83-0dc286a52ac9 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:14.736229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1196/2390] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:14.736949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b960f6f-6a4c-4ec2-8275-ba40715f4a27 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:14.737229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b960f6f-6a4c-4ec2-8275-ba40715f4a27 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:14.737501 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b960f6f-6a4c-4ec2-8275-ba40715f4a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:14.746583 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1195/2391] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:29:14.765188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-7b960f6f-6a4c-4ec2-8275-ba40715f4a27 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:14.766301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7b960f6f-6a4c-4ec2-8275-ba40715f4a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:14.778074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1197/2392] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:14.782223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d10ece-7c9e-4476-8c83-0dc286a52ac9 service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:14.812886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d10ece-7c9e-4476-8c83-0dc286a52ac9 service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:14.817615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:14.824467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1e77c-61c0-4e81-8f58-45039756a862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:14.827825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1e77c-61c0-4e81-8f58-45039756a862 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:14.827825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1e77c-61c0-4e81-8f58-45039756a862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:14.829268 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1e77c-61c0-4e81-8f58-45039756a862 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:14.830177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1e77c-61c0-4e81-8f58-45039756a862 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:29:14.832151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1e77c-61c0-4e81-8f58-45039756a862 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:14.839822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1e77c-61c0-4e81-8f58-45039756a862 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:14.878973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1196/2393] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:29:14.881930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b960f6f-6a4c-4ec2-8275-ba40715f4a27 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:14.897235 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b960f6f-6a4c-4ec2-8275-ba40715f4a27 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:14.910265 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a9fe-be57-4b08-9f52-7cb8975e92bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:14.910888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a9fe-be57-4b08-9f52-7cb8975e92bb admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:14.911168 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a9fe-be57-4b08-9f52-7cb8975e92bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:14.911427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a9fe-be57-4b08-9f52-7cb8975e92bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:14.912064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922a9fe-be57-4b08-9f52-7cb8975e92bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:14.916981 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922a9fe-be57-4b08-9f52-7cb8975e92bb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:15.099613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1198/2394] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:29:14 2022] POST /identity/v3/projects => generated 367 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:29:15.103721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1e77c-61c0-4e81-8f58-45039756a862 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:15.117393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1e77c-61c0-4e81-8f58-45039756a862 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:15.179722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bea94a-2117-425f-bfe2-b0c0ed73abdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:15.183371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bea94a-2117-425f-bfe2-b0c0ed73abdf admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:15.183371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bea94a-2117-425f-bfe2-b0c0ed73abdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:15.184955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bea94a-2117-425f-bfe2-b0c0ed73abdf admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:15.185729 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bea94a-2117-425f-bfe2-b0c0ed73abdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:15.187791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bea94a-2117-425f-bfe2-b0c0ed73abdf admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:15.315990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1197/2395] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:29:15 2022] POST /identity/v3/users => generated 284 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:29:15.320865 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a9fe-be57-4b08-9f52-7cb8975e92bb admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:15.333253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a9fe-be57-4b08-9f52-7cb8975e92bb admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:15.338173 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5cee7-7415-419e-8d3b-00ec66d54369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:15.339595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5cee7-7415-419e-8d3b-00ec66d54369 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:15.339879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5cee7-7415-419e-8d3b-00ec66d54369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:15.340151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5cee7-7415-419e-8d3b-00ec66d54369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:15.340871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5cee7-7415-419e-8d3b-00ec66d54369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:15.350091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5cee7-7415-419e-8d3b-00ec66d54369 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:15.362177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1199/2396] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:29:15.366047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bea94a-2117-425f-bfe2-b0c0ed73abdf admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:15.377492 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bea94a-2117-425f-bfe2-b0c0ed73abdf admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:15.383615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a499cb-6c76-4d88-b163-5d5213f83c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:15.384283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a499cb-6c76-4d88-b163-5d5213f83c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:15.384571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a499cb-6c76-4d88-b163-5d5213f83c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:15.384900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a499cb-6c76-4d88-b163-5d5213f83c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:15.386334 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a499cb-6c76-4d88-b163-5d5213f83c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:15.388557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a499cb-6c76-4d88-b163-5d5213f83c83 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:15.404830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1198/2397] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:29:15.406268 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5cee7-7415-419e-8d3b-00ec66d54369 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:15.420951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5cee7-7415-419e-8d3b-00ec66d54369 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:15.425807 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370ed1b-05d1-43c2-8427-f7d6d21252be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:15.426456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370ed1b-05d1-43c2-8427-f7d6d21252be admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:15.426731 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370ed1b-05d1-43c2-8427-f7d6d21252be admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:15.427005 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370ed1b-05d1-43c2-8427-f7d6d21252be admin admin] PATH_INFO: `/v3/projects/7d5ba78b4004447d8924041a2d80cd4d/users/a581e1220abb4f4bb578bda0f06a366c/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:15.445940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370ed1b-05d1-43c2-8427-f7d6d21252be admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5ba78b4004447d8924041a2d80cd4d, user_id=a581e1220abb4f4bb578bda0f06a366c, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:15.449425 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370ed1b-05d1-43c2-8427-f7d6d21252be admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:15.538522 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1200/2398] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:29:15 2022] PUT /identity/v3/projects/7d5ba78b4004447d8924041a2d80cd4d/users/a581e1220abb4f4bb578bda0f06a366c/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:15.549995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3acadd-a0b4-4c98-9d07-1bddbf1a2fd6 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:15.550515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3acadd-a0b4-4c98-9d07-1bddbf1a2fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:15.550750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3acadd-a0b4-4c98-9d07-1bddbf1a2fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:15.624890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7c3acadd-a0b4-4c98-9d07-1bddbf1a2fd6 None None] MFA Rules not processed for user `a581e1220abb4f4bb578bda0f06a366c`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:15.626149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7c3acadd-a0b4-4c98-9d07-1bddbf1a2fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:15.686519 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:15.718684 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1199/2399] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:15.731015 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370ed1b-05d1-43c2-8427-f7d6d21252be admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:15.737222 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c370ed1b-05d1-43c2-8427-f7d6d21252be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:15.815338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370ed1b-05d1-43c2-8427-f7d6d21252be admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:15.819345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0547d3a-d101-4215-88cc-74b6a36093f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7e768fdd1054e40b6b4910e39a6e533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:15.820255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0547d3a-d101-4215-88cc-74b6a36093f6 service glance] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:15.820706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0547d3a-d101-4215-88cc-74b6a36093f6 service glance] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:15.820982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0547d3a-d101-4215-88cc-74b6a36093f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:15.831074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0547d3a-d101-4215-88cc-74b6a36093f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:15.832669 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0547d3a-d101-4215-88cc-74b6a36093f6 service glance] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:15.847623 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1201/2400] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:29:16.991404 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3acadd-a0b4-4c98-9d07-1bddbf1a2fd6 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:16.994432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7c3acadd-a0b4-4c98-9d07-1bddbf1a2fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:17.054433 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3acadd-a0b4-4c98-9d07-1bddbf1a2fd6 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:17.058917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac053a-c0ef-4b95-9a21-6b788b10ed90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:17.059529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac053a-c0ef-4b95-9a21-6b788b10ed90 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:17.061094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac053a-c0ef-4b95-9a21-6b788b10ed90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:17.061094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac053a-c0ef-4b95-9a21-6b788b10ed90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:17.062090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-6cac053a-c0ef-4b95-9a21-6b788b10ed90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:17.124391 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac053a-c0ef-4b95-9a21-6b788b10ed90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:17.125877 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac053a-c0ef-4b95-9a21-6b788b10ed90 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:17.135147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1200/2401] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:29:17.303000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:19.012055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:19.459812 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0547d3a-d101-4215-88cc-74b6a36093f6 service glance] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:19.470090 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0547d3a-d101-4215-88cc-74b6a36093f6 service glance] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:19.475076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34bd4e-b89d-48d4-a5bb-9949a8fc6102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:19.475867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34bd4e-b89d-48d4-a5bb-9949a8fc6102 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:19.476340 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34bd4e-b89d-48d4-a5bb-9949a8fc6102 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:19.476887 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34bd4e-b89d-48d4-a5bb-9949a8fc6102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:19.479003 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-1f34bd4e-b89d-48d4-a5bb-9949a8fc6102 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:19.491774 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:19.546218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34bd4e-b89d-48d4-a5bb-9949a8fc6102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:19.547770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34bd4e-b89d-48d4-a5bb-9949a8fc6102 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:19.558093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1202/2402] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:29:20.537721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:20.577643 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:21.348876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:21.463693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:22.183383 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:22.903495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:23.662373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bb98b-3ab4-403e-a524-20da3ee8a5d8 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:23.662926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bb98b-3ab4-403e-a524-20da3ee8a5d8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:23.662926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bb98b-3ab4-403e-a524-20da3ee8a5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:23.681780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-2b9bb98b-3ab4-403e-a524-20da3ee8a5d8 None None] MFA Rules not processed for user `336345e5437341ba9859d776c44ea4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:23.682993 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2b9bb98b-3ab4-403e-a524-20da3ee8a5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:23.737203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1201/2403] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:23.811767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:24.426795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34bd4e-b89d-48d4-a5bb-9949a8fc6102 service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:24.428307 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-1f34bd4e-b89d-48d4-a5bb-9949a8fc6102 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:24.493459 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34bd4e-b89d-48d4-a5bb-9949a8fc6102 service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:24.498452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:24.508927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154e291-822a-4f04-832f-d3e9862e17bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:24.508927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154e291-822a-4f04-832f-d3e9862e17bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:24.508927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154e291-822a-4f04-832f-d3e9862e17bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:24.508927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154e291-822a-4f04-832f-d3e9862e17bb admin admin] PATH_INFO: `/v3/users/703acfc51abf4086bd65e48ec5541905` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:24.509495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154e291-822a-4f04-832f-d3e9862e17bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=703acfc51abf4086bd65e48ec5541905)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:24.509495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154e291-822a-4f04-832f-d3e9862e17bb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:24.556146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bb98b-3ab4-403e-a524-20da3ee8a5d8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:24.557752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2b9bb98b-3ab4-403e-a524-20da3ee8a5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:24.572670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bb98b-3ab4-403e-a524-20da3ee8a5d8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:24.577898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398bb58-f883-47d5-bea1-a417811b1c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:24.578559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398bb58-f883-47d5-bea1-a417811b1c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:24.578843 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398bb58-f883-47d5-bea1-a417811b1c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:24.579120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398bb58-f883-47d5-bea1-a417811b1c0c admin admin] PATH_INFO: `/v3/users/15150cfeaec8407187afddfd3d752fd9` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:24.580275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398bb58-f883-47d5-bea1-a417811b1c0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=15150cfeaec8407187afddfd3d752fd9)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:24.582271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398bb58-f883-47d5-bea1-a417811b1c0c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:24.837021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7154e291-822a-4f04-832f-d3e9862e17bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '703acfc51abf4086bd65e48ec5541905'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:24.843930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7154e291-822a-4f04-832f-d3e9862e17bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '703acfc51abf4086bd65e48ec5541905'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:24.859458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1398bb58-f883-47d5-bea1-a417811b1c0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15150cfeaec8407187afddfd3d752fd9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:24.860106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1398bb58-f883-47d5-bea1-a417811b1c0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15150cfeaec8407187afddfd3d752fd9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:24.866805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1398bb58-f883-47d5-bea1-a417811b1c0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15150cfeaec8407187afddfd3d752fd9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:24.954543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7154e291-822a-4f04-832f-d3e9862e17bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '703acfc51abf4086bd65e48ec5541905'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:24.955188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7154e291-822a-4f04-832f-d3e9862e17bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '703acfc51abf4086bd65e48ec5541905'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:24.967797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-7154e291-822a-4f04-832f-d3e9862e17bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '703acfc51abf4086bd65e48ec5541905'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:24.973799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1203/2404] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:29:24 2022] DELETE /identity/v3/users/703acfc51abf4086bd65e48ec5541905 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:24.978158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb4906-969e-42f0-8eec-0cbc077e9994 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:24.978455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb4906-969e-42f0-8eec-0cbc077e9994 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:24.978736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb4906-969e-42f0-8eec-0cbc077e9994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:24.998912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-c1bb4906-969e-42f0-8eec-0cbc077e9994 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:25.000149 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c1bb4906-969e-42f0-8eec-0cbc077e9994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:25.006000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1398bb58-f883-47d5-bea1-a417811b1c0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15150cfeaec8407187afddfd3d752fd9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:25.006620 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1398bb58-f883-47d5-bea1-a417811b1c0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15150cfeaec8407187afddfd3d752fd9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:25.025754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1202/2405] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:29:24 2022] DELETE /identity/v3/users/15150cfeaec8407187afddfd3d752fd9 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:29:25.030181 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257bf23-071a-4654-bbc9-42debf46e392 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:25.030467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257bf23-071a-4654-bbc9-42debf46e392 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:25.030745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257bf23-071a-4654-bbc9-42debf46e392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:25.051646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f257bf23-071a-4654-bbc9-42debf46e392 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:25.052961 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f257bf23-071a-4654-bbc9-42debf46e392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:25.060024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1204/2406] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:29:25.070206 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb4906-969e-42f0-8eec-0cbc077e9994 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:25.071591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c1bb4906-969e-42f0-8eec-0cbc077e9994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:25.116538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1203/2407] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:29:25.126957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257bf23-071a-4654-bbc9-42debf46e392 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:25.128441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f257bf23-071a-4654-bbc9-42debf46e392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:25.137703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb4906-969e-42f0-8eec-0cbc077e9994 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:25.178014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:25.188651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257bf23-071a-4654-bbc9-42debf46e392 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:25.188910 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a2ae0-5f3f-4bf9-9d86-87435b3b1ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:25.189537 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a2ae0-5f3f-4bf9-9d86-87435b3b1ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:25.189814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a2ae0-5f3f-4bf9-9d86-87435b3b1ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:25.190085 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a2ae0-5f3f-4bf9-9d86-87435b3b1ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:25.192365 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f603b4b-d20b-4dae-805a-7b3b5d0034ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:25.192994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f603b4b-d20b-4dae-805a-7b3b5d0034ca service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:25.193272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f603b4b-d20b-4dae-805a-7b3b5d0034ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:25.193549 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f603b4b-d20b-4dae-805a-7b3b5d0034ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:25.199787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a2ae0-5f3f-4bf9-9d86-87435b3b1ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:25.205698 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a2ae0-5f3f-4bf9-9d86-87435b3b1ee7 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:25.205914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f603b4b-d20b-4dae-805a-7b3b5d0034ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:25.205914 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f603b4b-d20b-4dae-805a-7b3b5d0034ca service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:25.210574 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1205/2408] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:29:25.213652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1204/2409] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:29:25.280353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:25.400296 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:25.474839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:25.582970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:25.611507 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:25.697415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:25.844052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a2ae0-5f3f-4bf9-9d86-87435b3b1ee7 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:25.844052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b61a2ae0-5f3f-4bf9-9d86-87435b3b1ee7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:25.850493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f603b4b-d20b-4dae-805a-7b3b5d0034ca service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:25.854611 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2f603b4b-d20b-4dae-805a-7b3b5d0034ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:25.867048 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:25.875999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a2ae0-5f3f-4bf9-9d86-87435b3b1ee7 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:25.884499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:25.885151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:25.885473 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:25.885754 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea admin admin] PATH_INFO: `/v3/projects/f9dc90d46bef473da35bdd69f71ae31d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:25.887078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9dc90d46bef473da35bdd69f71ae31d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:25.889342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:25.895284 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f603b4b-d20b-4dae-805a-7b3b5d0034ca service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:25.906464 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:25.907117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:25.908184 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:25.908484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] PATH_INFO: `/v3/projects/bb27b637f41448b6b18549867edb2319` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:25.910511 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb27b637f41448b6b18549867edb2319)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:25.912728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:25.923089 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:26.027893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9dc90d46bef473da35bdd69f71ae31d', 'request_id': 'req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.035078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9dc90d46bef473da35bdd69f71ae31d', 'request_id': 'req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.041380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9dc90d46bef473da35bdd69f71ae31d', 'request_id': 'req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.042355 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb27b637f41448b6b18549867edb2319', 'request_id': 'req-191e26cd-0359-4c5c-abba-2e539398a174'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.048361 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb27b637f41448b6b18549867edb2319', 'request_id': 'req-191e26cd-0359-4c5c-abba-2e539398a174'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.119549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea admin admin] The token cache is being invalidate because project f9dc90d46bef473da35bdd69f71ae31d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:29:26.120503 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.121127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-753d5a95-5e7d-4ebd-8d73-d8cf112e08ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.122598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1206/2410] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:29:25 2022] DELETE /identity/v3/projects/f9dc90d46bef473da35bdd69f71ae31d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:26.127362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9fa46-62c4-461e-8136-9fa76e93add9 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:26.127663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9fa46-62c4-461e-8136-9fa76e93add9 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:26.127937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9fa46-62c4-461e-8136-9fa76e93add9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:26.150433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3dc9fa46-62c4-461e-8136-9fa76e93add9 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:26.151718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3dc9fa46-62c4-461e-8136-9fa76e93add9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:26.194196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb27b637f41448b6b18549867edb2319', 'request_id': 'req-191e26cd-0359-4c5c-abba-2e539398a174'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.206241 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] The token cache is being invalidate because project bb27b637f41448b6b18549867edb2319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:29:26.208252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.208252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.208911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1205/2411] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:29:25 2022] DELETE /identity/v3/projects/bb27b637f41448b6b18549867edb2319 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:26.212635 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:26.212851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:26.214564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:26.216180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1207/2412] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:29:26.223729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e04c2-fb86-4dcf-8d62-c763a7ab5190 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:26.224016 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e04c2-fb86-4dcf-8d62-c763a7ab5190 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:26.224318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e04c2-fb86-4dcf-8d62-c763a7ab5190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:26.227802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e26cd-0359-4c5c-abba-2e539398a174 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:26.233107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:26.238061 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:26.238061 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:26.238061 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] PATH_INFO: `/v3/users/1ce50a8460864cd4b5fa5fb0d901d166` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:26.238061 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ce50a8460864cd4b5fa5fb0d901d166)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:26.238061 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:26.245458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-4a3e04c2-fb86-4dcf-8d62-c763a7ab5190 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:26.246659 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-4a3e04c2-fb86-4dcf-8d62-c763a7ab5190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:26.254751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1208/2413] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:29:26.258952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e04c2-fb86-4dcf-8d62-c763a7ab5190 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:26.268406 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e04c2-fb86-4dcf-8d62-c763a7ab5190 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:26.273264 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557df3a-c30a-45ad-bd36-7f9e743b358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:26.278143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557df3a-c30a-45ad-bd36-7f9e743b358f admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:26.278143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557df3a-c30a-45ad-bd36-7f9e743b358f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:26.278143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557df3a-c30a-45ad-bd36-7f9e743b358f admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:26.278143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557df3a-c30a-45ad-bd36-7f9e743b358f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:29:26.278143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557df3a-c30a-45ad-bd36-7f9e743b358f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:26.278143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557df3a-c30a-45ad-bd36-7f9e743b358f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:26.288838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1209/2414] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:29:26.293709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557df3a-c30a-45ad-bd36-7f9e743b358f admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:26.297135 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:26.302817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557df3a-c30a-45ad-bd36-7f9e743b358f admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:26.308030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e085383-f7e5-45d1-bc17-6dfa942ea8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:26.308610 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e085383-f7e5-45d1-bc17-6dfa942ea8c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:26.308890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e085383-f7e5-45d1-bc17-6dfa942ea8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:26.309160 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e085383-f7e5-45d1-bc17-6dfa942ea8c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:26.309797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e085383-f7e5-45d1-bc17-6dfa942ea8c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:26.311639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e085383-f7e5-45d1-bc17-6dfa942ea8c9 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:26.403451 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ce50a8460864cd4b5fa5fb0d901d166'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.404157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ce50a8460864cd4b5fa5fb0d901d166'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.410318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1210/2415] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:29:26 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:29:26.411092 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e085383-f7e5-45d1-bc17-6dfa942ea8c9 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:26.411344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ce50a8460864cd4b5fa5fb0d901d166'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.476476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e085383-f7e5-45d1-bc17-6dfa942ea8c9 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:26.482251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4eed8-898c-4159-ba13-026f8bd2630a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:26.482858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4eed8-898c-4159-ba13-026f8bd2630a admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:26.483140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4eed8-898c-4159-ba13-026f8bd2630a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:26.483466 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4eed8-898c-4159-ba13-026f8bd2630a admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:26.484099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4eed8-898c-4159-ba13-026f8bd2630a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:26.486068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4eed8-898c-4159-ba13-026f8bd2630a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:26.497841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ce50a8460864cd4b5fa5fb0d901d166'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.498751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ce50a8460864cd4b5fa5fb0d901d166'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:26.519979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1206/2416] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:29:26 2022] DELETE /identity/v3/users/1ce50a8460864cd4b5fa5fb0d901d166 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:29:26.586701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1211/2417] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:29:26 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:29:26.587981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:26.589919 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:26.628872 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:26.667225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6eaf28-92f0-4659-b159-560f9e8d2904 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:26.674403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721879f-a524-4c36-90c4-1681883347f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:26.674403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721879f-a524-4c36-90c4-1681883347f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:26.674403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721879f-a524-4c36-90c4-1681883347f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:26.674403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721879f-a524-4c36-90c4-1681883347f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:26.674861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721879f-a524-4c36-90c4-1681883347f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:26.675941 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721879f-a524-4c36-90c4-1681883347f1 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:26.686493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1207/2418] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:29:26.690632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4eed8-898c-4159-ba13-026f8bd2630a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:26.701351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4eed8-898c-4159-ba13-026f8bd2630a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:26.706074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20cb9d-5371-4858-9c06-0fadd151d72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:26.706662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20cb9d-5371-4858-9c06-0fadd151d72a admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:26.706937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20cb9d-5371-4858-9c06-0fadd151d72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:26.707230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20cb9d-5371-4858-9c06-0fadd151d72a admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:26.708015 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20cb9d-5371-4858-9c06-0fadd151d72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:26.709672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20cb9d-5371-4858-9c06-0fadd151d72a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:26.717782 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:26.720603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1212/2419] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:29:26.731344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721879f-a524-4c36-90c4-1681883347f1 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:26.749023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721879f-a524-4c36-90c4-1681883347f1 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:26.754536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d442c-b3c3-4af3-90f0-c8df97e2b70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:26.755371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d442c-b3c3-4af3-90f0-c8df97e2b70f admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:26.755750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d442c-b3c3-4af3-90f0-c8df97e2b70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:26.756617 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d442c-b3c3-4af3-90f0-c8df97e2b70f admin admin] PATH_INFO: `/v3/projects/0931206111ba46198d58746f57266c23/users/e82c9e9f4ca742abb6531b3210be3776/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:26.772452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d442c-b3c3-4af3-90f0-c8df97e2b70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0931206111ba46198d58746f57266c23, user_id=e82c9e9f4ca742abb6531b3210be3776, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:26.775116 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d442c-b3c3-4af3-90f0-c8df97e2b70f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:26.910126 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1208/2420] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:29:26 2022] PUT /identity/v3/projects/0931206111ba46198d58746f57266c23/users/e82c9e9f4ca742abb6531b3210be3776/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:26.915413 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de870552-2c0f-435a-913b-49c052a77235 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:26.915702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de870552-2c0f-435a-913b-49c052a77235 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:26.915975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de870552-2c0f-435a-913b-49c052a77235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:26.935990 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-de870552-2c0f-435a-913b-49c052a77235 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:26.937230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-de870552-2c0f-435a-913b-49c052a77235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:26.999191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1213/2421] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:27.009074 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d442c-b3c3-4af3-90f0-c8df97e2b70f admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:27.011916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-db1d442c-b3c3-4af3-90f0-c8df97e2b70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:27.026836 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de870552-2c0f-435a-913b-49c052a77235 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:27.028826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-de870552-2c0f-435a-913b-49c052a77235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:27.041523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de870552-2c0f-435a-913b-49c052a77235 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:27.046503 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:27.047133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:27.047410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:27.047960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] PATH_INFO: `/v3/projects/db3764fd345b476fa94f5f68d1084e15` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:27.049387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] RBAC: Authorizing `identity:delete_project(project_id=db3764fd345b476fa94f5f68d1084e15)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:27.051528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:27.080623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d442c-b3c3-4af3-90f0-c8df97e2b70f admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:27.084684 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d9784-d405-4459-a21c-7d3756f12425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:27.085278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9784-d405-4459-a21c-7d3756f12425 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:27.085574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9784-d405-4459-a21c-7d3756f12425 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:27.085844 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9784-d405-4459-a21c-7d3756f12425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:27.095239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d9784-d405-4459-a21c-7d3756f12425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:27.096752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d9784-d405-4459-a21c-7d3756f12425 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:27.106543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1209/2422] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:29:27.206863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db3764fd345b476fa94f5f68d1084e15', 'request_id': 'req-486845af-71d5-4976-90f1-793c7d16f83e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:27.213819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db3764fd345b476fa94f5f68d1084e15', 'request_id': 'req-486845af-71d5-4976-90f1-793c7d16f83e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:27.219394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:27.222006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db3764fd345b476fa94f5f68d1084e15', 'request_id': 'req-486845af-71d5-4976-90f1-793c7d16f83e'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:27.298483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:27.315244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] The token cache is being invalidate because project db3764fd345b476fa94f5f68d1084e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:29:27.315602 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:27.316315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:27.317897 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1214/2423] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:29:27 2022] DELETE /identity/v3/projects/db3764fd345b476fa94f5f68d1084e15 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:27.321865 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d9784-d405-4459-a21c-7d3756f12425 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:27.331973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d9784-d405-4459-a21c-7d3756f12425 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:27.388770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d5e44-2d97-47a0-ba9d-2bb933d1c5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:27.389409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d5e44-2d97-47a0-ba9d-2bb933d1c5c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:27.389688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d5e44-2d97-47a0-ba9d-2bb933d1c5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:27.390088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d5e44-2d97-47a0-ba9d-2bb933d1c5c4 admin admin] PATH_INFO: `/v3/users/d363e1a47cf247a0a7c4aeeba9f408ac` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:27.391303 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d5e44-2d97-47a0-ba9d-2bb933d1c5c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d363e1a47cf247a0a7c4aeeba9f408ac)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:27.393416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d5e44-2d97-47a0-ba9d-2bb933d1c5c4 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:27.637415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-145d5e44-2d97-47a0-ba9d-2bb933d1c5c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd363e1a47cf247a0a7c4aeeba9f408ac'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:27.638087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-145d5e44-2d97-47a0-ba9d-2bb933d1c5c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd363e1a47cf247a0a7c4aeeba9f408ac'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:27.645476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-145d5e44-2d97-47a0-ba9d-2bb933d1c5c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd363e1a47cf247a0a7c4aeeba9f408ac'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:27.720784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-145d5e44-2d97-47a0-ba9d-2bb933d1c5c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd363e1a47cf247a0a7c4aeeba9f408ac'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:27.721629 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-145d5e44-2d97-47a0-ba9d-2bb933d1c5c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd363e1a47cf247a0a7c4aeeba9f408ac'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:27.742780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1210/2424] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:29:27 2022] DELETE /identity/v3/users/d363e1a47cf247a0a7c4aeeba9f408ac => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:29:28.189172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:28.234623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:28.645583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:28.648330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:28.662031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486845af-71d5-4976-90f1-793c7d16f83e admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:28.735025 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:28.735851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:28.736135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:28.736467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] PATH_INFO: `/v3/projects/7ba4691d0f3f4a3f9dcf72a8b385a829` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:28.737651 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ba4691d0f3f4a3f9dcf72a8b385a829)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:28.739708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:28.829061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ba4691d0f3f4a3f9dcf72a8b385a829', 'request_id': 'req-4a67be30-e77c-4de5-996c-7835c5755040'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:28.835773 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ba4691d0f3f4a3f9dcf72a8b385a829', 'request_id': 'req-4a67be30-e77c-4de5-996c-7835c5755040'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:28.843298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ba4691d0f3f4a3f9dcf72a8b385a829', 'request_id': 'req-4a67be30-e77c-4de5-996c-7835c5755040'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:28.897807 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:28.971264 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] The token cache is being invalidate because project 7ba4691d0f3f4a3f9dcf72a8b385a829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:29:28.971666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:28.973663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:28.975252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1215/2425] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:29:28 2022] DELETE /identity/v3/projects/7ba4691d0f3f4a3f9dcf72a8b385a829 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:29.035243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:30.071761 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:30.098259 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:30.691725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:30.835705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:31.120432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:31.497542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:31.577671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:31.837749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:31.960567 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:32.152372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:32.235198 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:32.600906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:32.847915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:33.068799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:33.269210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:33.326437 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:33.411752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95b055-1a81-4208-9f7a-e9077cacac9a None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:33.412044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95b055-1a81-4208-9f7a-e9077cacac9a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:33.412375 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95b055-1a81-4208-9f7a-e9077cacac9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:33.443926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-3b95b055-1a81-4208-9f7a-e9077cacac9a None None] MFA Rules not processed for user `e82c9e9f4ca742abb6531b3210be3776`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:33.445261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3b95b055-1a81-4208-9f7a-e9077cacac9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:33.515053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1211/2426] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:33.519948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:33.521190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:33.573142 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:33.592007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67be30-e77c-4de5-996c-7835c5755040 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:33.596760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0998f1-0552-4b6b-a1ea-93a141444687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:33.597374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0998f1-0552-4b6b-a1ea-93a141444687 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:33.597686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0998f1-0552-4b6b-a1ea-93a141444687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:33.597963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0998f1-0552-4b6b-a1ea-93a141444687 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:33.598611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0998f1-0552-4b6b-a1ea-93a141444687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:33.600606 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0998f1-0552-4b6b-a1ea-93a141444687 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:33.607402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:33.782984 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1216/2427] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:29:33 2022] POST /identity/v3/projects => generated 385 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:29:33.787313 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95b055-1a81-4208-9f7a-e9077cacac9a None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:33.798594 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95b055-1a81-4208-9f7a-e9077cacac9a None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:33.863075 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcb07a-c766-4793-b1dd-5ba596347563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:33.865344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcb07a-c766-4793-b1dd-5ba596347563 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:33.865344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcb07a-c766-4793-b1dd-5ba596347563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:33.865344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcb07a-c766-4793-b1dd-5ba596347563 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:33.865344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fcb07a-c766-4793-b1dd-5ba596347563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:33.866924 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fcb07a-c766-4793-b1dd-5ba596347563 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:33.886484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:33.949964 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1212/2428] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:29:33 2022] POST /identity/v3/users => generated 299 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:29:33.954197 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0998f1-0552-4b6b-a1ea-93a141444687 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:33.966618 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0998f1-0552-4b6b-a1ea-93a141444687 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:33.972448 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5eead-852b-410b-8b56-a7c4f842a532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:33.973068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5eead-852b-410b-8b56-a7c4f842a532 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:33.973352 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5eead-852b-410b-8b56-a7c4f842a532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:33.973626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5eead-852b-410b-8b56-a7c4f842a532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:33.974306 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5eead-852b-410b-8b56-a7c4f842a532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:33.976224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5eead-852b-410b-8b56-a7c4f842a532 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:33.988769 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1217/2429] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:29:33.993994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcb07a-c766-4793-b1dd-5ba596347563 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:34.006475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcb07a-c766-4793-b1dd-5ba596347563 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:34.011772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c771d-c91f-43ec-977f-b72fd9f0aa2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:34.012520 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c771d-c91f-43ec-977f-b72fd9f0aa2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:34.012825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c771d-c91f-43ec-977f-b72fd9f0aa2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:34.013093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c771d-c91f-43ec-977f-b72fd9f0aa2b admin admin] PATH_INFO: `/v3/projects/789d1fa851cf429a8752fd7df78a8932/users/0f670b6b79d24a679364ab356fbc9562/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:34.030337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c771d-c91f-43ec-977f-b72fd9f0aa2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=789d1fa851cf429a8752fd7df78a8932, user_id=0f670b6b79d24a679364ab356fbc9562, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:34.033084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c771d-c91f-43ec-977f-b72fd9f0aa2b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:34.100871 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1213/2430] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:29:33 2022] PUT /identity/v3/projects/789d1fa851cf429a8752fd7df78a8932/users/0f670b6b79d24a679364ab356fbc9562/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:34.103159 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:34.105420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5eead-852b-410b-8b56-a7c4f842a532 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:34.115477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5eead-852b-410b-8b56-a7c4f842a532 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:34.176278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5bd79b-2e06-4b85-bcc4-dff21c1c38c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:34.177017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bd79b-2e06-4b85-bcc4-dff21c1c38c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:34.177393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bd79b-2e06-4b85-bcc4-dff21c1c38c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:34.177605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:34.177757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bd79b-2e06-4b85-bcc4-dff21c1c38c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:34.179716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bd79b-2e06-4b85-bcc4-dff21c1c38c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:34.181512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bd79b-2e06-4b85-bcc4-dff21c1c38c4 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:34.195770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1218/2431] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:29:34.197731 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c771d-c91f-43ec-977f-b72fd9f0aa2b admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:34.208661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c771d-c91f-43ec-977f-b72fd9f0aa2b admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:34.213657 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78425abc-d3f5-4c03-ba1f-f57c6dbcc52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:34.214320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78425abc-d3f5-4c03-ba1f-f57c6dbcc52c admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:34.214606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78425abc-d3f5-4c03-ba1f-f57c6dbcc52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:34.214883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78425abc-d3f5-4c03-ba1f-f57c6dbcc52c admin admin] PATH_INFO: `/v3/projects/789d1fa851cf429a8752fd7df78a8932/users/0f670b6b79d24a679364ab356fbc9562/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:34.216920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78425abc-d3f5-4c03-ba1f-f57c6dbcc52c admin admin] RBAC: Authorizing `identity:create_grant(project_id=789d1fa851cf429a8752fd7df78a8932, user_id=0f670b6b79d24a679364ab356fbc9562, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:34.219610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78425abc-d3f5-4c03-ba1f-f57c6dbcc52c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:34.231125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1214/2432] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:29:34 2022] PUT /identity/v3/projects/789d1fa851cf429a8752fd7df78a8932/users/0f670b6b79d24a679364ab356fbc9562/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:34.570411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:34.617751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:35.383285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:36.082063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:36.240539 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:37.035802 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:37.236936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:38.517176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:38.989086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:40.131751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:41.196399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0586f4-81a7-4440-921b-86802ed224aa None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:41.196951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0586f4-81a7-4440-921b-86802ed224aa None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:41.196951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0586f4-81a7-4440-921b-86802ed224aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:41.212420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:41.231835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ec0586f4-81a7-4440-921b-86802ed224aa None None] MFA Rules not processed for user `0f670b6b79d24a679364ab356fbc9562`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:41.233107 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ec0586f4-81a7-4440-921b-86802ed224aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:41.315822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1219/2433] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:41.323410 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78425abc-d3f5-4c03-ba1f-f57c6dbcc52c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:41.325201 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-78425abc-d3f5-4c03-ba1f-f57c6dbcc52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:41.387120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78425abc-d3f5-4c03-ba1f-f57c6dbcc52c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:41.399645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf36dd-20d3-4ae0-9e2c-5c068a96ee6f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7e768fdd1054e40b6b4910e39a6e533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:41.401940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf36dd-20d3-4ae0-9e2c-5c068a96ee6f service glance] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:41.401940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf36dd-20d3-4ae0-9e2c-5c068a96ee6f service glance] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:41.401940 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf36dd-20d3-4ae0-9e2c-5c068a96ee6f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:41.411301 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:41.465371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf36dd-20d3-4ae0-9e2c-5c068a96ee6f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:41.467050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf36dd-20d3-4ae0-9e2c-5c068a96ee6f service glance] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:41.477961 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1215/2434] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:29:41.813278 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debb22b-c770-4a5a-a3df-5decb68e32c6 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:41.813571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debb22b-c770-4a5a-a3df-5decb68e32c6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:41.813850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debb22b-c770-4a5a-a3df-5decb68e32c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:41.838373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-2debb22b-c770-4a5a-a3df-5decb68e32c6 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:41.839841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2debb22b-c770-4a5a-a3df-5decb68e32c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:41.904356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1220/2435] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:29:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:41.955421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf36dd-20d3-4ae0-9e2c-5c068a96ee6f service glance] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:41.956931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e0cf36dd-20d3-4ae0-9e2c-5c068a96ee6f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:42.030697 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf36dd-20d3-4ae0-9e2c-5c068a96ee6f service glance] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:42.037673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37090cb7-0b3e-4d06-a3c5-9d3b39c68de3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:42.038294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37090cb7-0b3e-4d06-a3c5-9d3b39c68de3 service swift] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:42.038863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37090cb7-0b3e-4d06-a3c5-9d3b39c68de3 service swift] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:42.039141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37090cb7-0b3e-4d06-a3c5-9d3b39c68de3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:42.049529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37090cb7-0b3e-4d06-a3c5-9d3b39c68de3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:42.052258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37090cb7-0b3e-4d06-a3c5-9d3b39c68de3 service swift] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:42.063684 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1216/2436] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:29:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:29:52.059811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=56 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:52.229082 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=55 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:55.364522 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a98e5-504d-46cf-b19a-512bc0478c14 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:55.365088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a98e5-504d-46cf-b19a-512bc0478c14 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:55.365088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a98e5-504d-46cf-b19a-512bc0478c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:55.386042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b60a98e5-504d-46cf-b19a-512bc0478c14 None None] MFA Rules not processed for user `e82c9e9f4ca742abb6531b3210be3776`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:55.387561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b60a98e5-504d-46cf-b19a-512bc0478c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:55.402095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1221/2437] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:29:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:03.685885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5190749-61e4-4a5d-b032-fda28b542b53 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:03.686422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5190749-61e4-4a5d-b032-fda28b542b53 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:03.686520 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5190749-61e4-4a5d-b032-fda28b542b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:03.708019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-d5190749-61e4-4a5d-b032-fda28b542b53 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:03.709346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-d5190749-61e4-4a5d-b032-fda28b542b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:03.716616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1217/2438] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:30:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:03.728801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a98e5-504d-46cf-b19a-512bc0478c14 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:03.741270 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a98e5-504d-46cf-b19a-512bc0478c14 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:03.751925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e427157-f5dc-4edc-a155-092e673f6c50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:03.751925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427157-f5dc-4edc-a155-092e673f6c50 service swift] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:03.751925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427157-f5dc-4edc-a155-092e673f6c50 service swift] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:03.751925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427157-f5dc-4edc-a155-092e673f6c50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:03.756756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e427157-f5dc-4edc-a155-092e673f6c50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:03.758434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e427157-f5dc-4edc-a155-092e673f6c50 service swift] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:03.767450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1222/2439] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:30:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:30:08.275162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5190749-61e4-4a5d-b032-fda28b542b53 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:08.277286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-d5190749-61e4-4a5d-b032-fda28b542b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:08.363588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5190749-61e4-4a5d-b032-fda28b542b53 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:08.368084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:08.369271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:08.369598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:08.369893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] PATH_INFO: `/v3/users/a581e1220abb4f4bb578bda0f06a366c` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:08.371014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a581e1220abb4f4bb578bda0f06a366c)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:08.372951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:08.577381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a581e1220abb4f4bb578bda0f06a366c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:30:08.578222 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a581e1220abb4f4bb578bda0f06a366c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:30:08.593631 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a581e1220abb4f4bb578bda0f06a366c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:30:08.673947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a581e1220abb4f4bb578bda0f06a366c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:30:08.675036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a581e1220abb4f4bb578bda0f06a366c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:30:08.697166 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1218/2440] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:30:08 2022] DELETE /identity/v3/users/a581e1220abb4f4bb578bda0f06a366c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:30:08.702362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7822e0-769b-4b7e-9fa3-d10158077c84 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:08.702925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7822e0-769b-4b7e-9fa3-d10158077c84 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:08.703039 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7822e0-769b-4b7e-9fa3-d10158077c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:08.723997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-9f7822e0-769b-4b7e-9fa3-d10158077c84 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:08.725872 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9f7822e0-769b-4b7e-9fa3-d10158077c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:08.800445 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1223/2441] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:08.811293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:08.814432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:08.887090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896df92-f361-4bb5-bc9e-5dcb8cded925 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:08.891085 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621511f-d0c7-46c3-b6d5-475d2d29a7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:08.891938 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621511f-d0c7-46c3-b6d5-475d2d29a7df service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:08.892346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621511f-d0c7-46c3-b6d5-475d2d29a7df service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:08.892693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621511f-d0c7-46c3-b6d5-475d2d29a7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:08.903502 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621511f-d0c7-46c3-b6d5-475d2d29a7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:08.905049 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621511f-d0c7-46c3-b6d5-475d2d29a7df service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:08.916123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1219/2442] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:09.411309 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7822e0-769b-4b7e-9fa3-d10158077c84 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:09.412803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9f7822e0-769b-4b7e-9fa3-d10158077c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:09.450611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7822e0-769b-4b7e-9fa3-d10158077c84 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:09.456808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc53c87-c43d-40e3-9923-4ff823e53823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:09.457465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc53c87-c43d-40e3-9923-4ff823e53823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:09.457749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc53c87-c43d-40e3-9923-4ff823e53823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:09.458020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc53c87-c43d-40e3-9923-4ff823e53823 admin admin] PATH_INFO: `/v3/projects/7d5ba78b4004447d8924041a2d80cd4d` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:09.460627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc53c87-c43d-40e3-9923-4ff823e53823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d5ba78b4004447d8924041a2d80cd4d)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:09.463455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc53c87-c43d-40e3-9923-4ff823e53823 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:09.648794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bbc53c87-c43d-40e3-9923-4ff823e53823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d5ba78b4004447d8924041a2d80cd4d', 'request_id': 'req-bbc53c87-c43d-40e3-9923-4ff823e53823'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:30:09.658828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bbc53c87-c43d-40e3-9923-4ff823e53823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d5ba78b4004447d8924041a2d80cd4d', 'request_id': 'req-bbc53c87-c43d-40e3-9923-4ff823e53823'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:30:09.666585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bbc53c87-c43d-40e3-9923-4ff823e53823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d5ba78b4004447d8924041a2d80cd4d', 'request_id': 'req-bbc53c87-c43d-40e3-9923-4ff823e53823'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:30:09.771144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bbc53c87-c43d-40e3-9923-4ff823e53823 admin admin] The token cache is being invalidate because project 7d5ba78b4004447d8924041a2d80cd4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:30:09.771781 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bbc53c87-c43d-40e3-9923-4ff823e53823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:30:09.772344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bbc53c87-c43d-40e3-9923-4ff823e53823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:30:09.773967 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1224/2443] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:30:09 2022] DELETE /identity/v3/projects/7d5ba78b4004447d8924041a2d80cd4d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:30:09.779322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed633fe-2c8a-4c0f-9c59-01bc31a7d345 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:09.784000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed633fe-2c8a-4c0f-9c59-01bc31a7d345 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:09.784000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed633fe-2c8a-4c0f-9c59-01bc31a7d345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:09.806114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-6ed633fe-2c8a-4c0f-9c59-01bc31a7d345 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:09.807598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-6ed633fe-2c8a-4c0f-9c59-01bc31a7d345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:09.818974 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:30:09.880033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1220/2444] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:30:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:09.885255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48ad1a-917a-418e-a6e2-e9749ec45426 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:09.885538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48ad1a-917a-418e-a6e2-e9749ec45426 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:09.885809 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48ad1a-917a-418e-a6e2-e9749ec45426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:09.907884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3f48ad1a-917a-418e-a6e2-e9749ec45426 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:09.909202 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3f48ad1a-917a-418e-a6e2-e9749ec45426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:09.917991 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1225/2445] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:30:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:09.922304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed633fe-2c8a-4c0f-9c59-01bc31a7d345 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:09.934426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed633fe-2c8a-4c0f-9c59-01bc31a7d345 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:09.950271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8f076-7169-412a-a6dd-fdb8edad8a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:09.950893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f076-7169-412a-a6dd-fdb8edad8a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:09.951173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f076-7169-412a-a6dd-fdb8edad8a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:09.951441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f076-7169-412a-a6dd-fdb8edad8a2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:09.952106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8f076-7169-412a-a6dd-fdb8edad8a2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:30:09.952433 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8f076-7169-412a-a6dd-fdb8edad8a2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:09.954057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8f076-7169-412a-a6dd-fdb8edad8a2e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:09.966724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1221/2446] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:30:09.971220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48ad1a-917a-418e-a6e2-e9749ec45426 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:09.981982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48ad1a-917a-418e-a6e2-e9749ec45426 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:09.986809 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cbc09-13b2-47d0-b3e8-558cab31d50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:09.987417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cbc09-13b2-47d0-b3e8-558cab31d50a admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:09.987700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cbc09-13b2-47d0-b3e8-558cab31d50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:09.987974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cbc09-13b2-47d0-b3e8-558cab31d50a admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:09.988644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cbc09-13b2-47d0-b3e8-558cab31d50a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:09.990571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cbc09-13b2-47d0-b3e8-558cab31d50a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:10.076840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8f076-7169-412a-a6dd-fdb8edad8a2e admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:10.077624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1226/2447] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:30:09 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:30:10.087185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8f076-7169-412a-a6dd-fdb8edad8a2e admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:10.149751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565096ac-a96f-4d5a-b3f8-f796f69fb476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:10.151157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565096ac-a96f-4d5a-b3f8-f796f69fb476 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:10.151556 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565096ac-a96f-4d5a-b3f8-f796f69fb476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:10.152023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565096ac-a96f-4d5a-b3f8-f796f69fb476 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:10.152854 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565096ac-a96f-4d5a-b3f8-f796f69fb476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:10.155063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565096ac-a96f-4d5a-b3f8-f796f69fb476 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:10.251833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1222/2448] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:30:10 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:30:10.255815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cbc09-13b2-47d0-b3e8-558cab31d50a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:10.269351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cbc09-13b2-47d0-b3e8-558cab31d50a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:10.281657 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea716545-0411-4b7f-86f9-91f223012e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:10.282354 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea716545-0411-4b7f-86f9-91f223012e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:10.282633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea716545-0411-4b7f-86f9-91f223012e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:10.282912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea716545-0411-4b7f-86f9-91f223012e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:10.283895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea716545-0411-4b7f-86f9-91f223012e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:10.285967 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea716545-0411-4b7f-86f9-91f223012e15 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:10.302946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1227/2449] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:30:10.304143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565096ac-a96f-4d5a-b3f8-f796f69fb476 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:10.314421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565096ac-a96f-4d5a-b3f8-f796f69fb476 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:10.319128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10d15f-cbe5-4803-9684-2087cd948569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:10.319730 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10d15f-cbe5-4803-9684-2087cd948569 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:10.320008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10d15f-cbe5-4803-9684-2087cd948569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:10.320310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10d15f-cbe5-4803-9684-2087cd948569 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:10.320951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10d15f-cbe5-4803-9684-2087cd948569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:10.323252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10d15f-cbe5-4803-9684-2087cd948569 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:10.338564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1223/2450] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:30:10.343548 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea716545-0411-4b7f-86f9-91f223012e15 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:10.357101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea716545-0411-4b7f-86f9-91f223012e15 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:10.362949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311f317-d667-4e36-ab74-c6dcc7d084ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:10.363615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311f317-d667-4e36-ab74-c6dcc7d084ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:10.363900 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311f317-d667-4e36-ab74-c6dcc7d084ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:10.364308 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311f317-d667-4e36-ab74-c6dcc7d084ca admin admin] PATH_INFO: `/v3/projects/ac65a0d70d7045d183a8a725b304c169/users/83efc8c932d04432ad3d0c974e0333a9/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:10.380759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311f317-d667-4e36-ab74-c6dcc7d084ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac65a0d70d7045d183a8a725b304c169, user_id=83efc8c932d04432ad3d0c974e0333a9, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:10.383690 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311f317-d667-4e36-ab74-c6dcc7d084ca admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:10.475791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:30:10.481671 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1228/2451] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:30:10 2022] PUT /identity/v3/projects/ac65a0d70d7045d183a8a725b304c169/users/83efc8c932d04432ad3d0c974e0333a9/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:30:10.486713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44163b94-217b-433f-ba9f-7ed55e7da710 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:10.487060 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44163b94-217b-433f-ba9f-7ed55e7da710 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:10.487402 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44163b94-217b-433f-ba9f-7ed55e7da710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:10.508817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-44163b94-217b-433f-ba9f-7ed55e7da710 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:10.510062 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-44163b94-217b-433f-ba9f-7ed55e7da710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:10.589446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1224/2452] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:30:10.599895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311f317-d667-4e36-ab74-c6dcc7d084ca admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:10.602780 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-6311f317-d667-4e36-ab74-c6dcc7d084ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:10.684244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311f317-d667-4e36-ab74-c6dcc7d084ca admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:10.687998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cd96f-d7f9-4658-bb4d-eca1de3047f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:10.688646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cd96f-d7f9-4658-bb4d-eca1de3047f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:10.688930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cd96f-d7f9-4658-bb4d-eca1de3047f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:10.689209 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cd96f-d7f9-4658-bb4d-eca1de3047f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:10.699438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6cd96f-d7f9-4658-bb4d-eca1de3047f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:10.700932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6cd96f-d7f9-4658-bb4d-eca1de3047f6 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:10.713847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1229/2453] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:30:12.332525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6a83e-09e5-4c00-80df-69240f2499ab None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:12.333338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6a83e-09e5-4c00-80df-69240f2499ab None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:12.333338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6a83e-09e5-4c00-80df-69240f2499ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:12.385235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f7c6a83e-09e5-4c00-80df-69240f2499ab None None] MFA Rules not processed for user `83efc8c932d04432ad3d0c974e0333a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:12.387090 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f7c6a83e-09e5-4c00-80df-69240f2499ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:12.401114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:30:12.474033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1225/2454] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:30:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:12.491878 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cd96f-d7f9-4658-bb4d-eca1de3047f6 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:12.493789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9f6cd96f-d7f9-4658-bb4d-eca1de3047f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:12.565920 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cd96f-d7f9-4658-bb4d-eca1de3047f6 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:12.571397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fccc50-15b0-460e-847c-f3feb468e7a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:12.571397 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fccc50-15b0-460e-847c-f3feb468e7a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:12.572261 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fccc50-15b0-460e-847c-f3feb468e7a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:12.572455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fccc50-15b0-460e-847c-f3feb468e7a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:12.582137 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fccc50-15b0-460e-847c-f3feb468e7a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:12.584231 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fccc50-15b0-460e-847c-f3feb468e7a1 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:12.596478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1230/2455] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:16.798057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a852431-cd5c-402e-bf95-f289d27c75ae None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:16.798643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a852431-cd5c-402e-bf95-f289d27c75ae None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:16.798643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a852431-cd5c-402e-bf95-f289d27c75ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:16.851036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7a852431-cd5c-402e-bf95-f289d27c75ae None None] MFA Rules not processed for user `83efc8c932d04432ad3d0c974e0333a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:16.853436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7a852431-cd5c-402e-bf95-f289d27c75ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:16.866824 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1226/2456] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:30:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:18.017872 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fccc50-15b0-460e-847c-f3feb468e7a1 service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:18.020227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-56fccc50-15b0-460e-847c-f3feb468e7a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:18.248931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fccc50-15b0-460e-847c-f3feb468e7a1 service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:18.254331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863c53c-2f39-4bef-bbef-e5231fb9f7e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:18.254976 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863c53c-2f39-4bef-bbef-e5231fb9f7e9 service placement] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:18.255315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863c53c-2f39-4bef-bbef-e5231fb9f7e9 service placement] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:18.255749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863c53c-2f39-4bef-bbef-e5231fb9f7e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:18.257955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7863c53c-2f39-4bef-bbef-e5231fb9f7e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:18.283646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863c53c-2f39-4bef-bbef-e5231fb9f7e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:18.285406 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863c53c-2f39-4bef-bbef-e5231fb9f7e9 service placement] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:18.299767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1231/2457] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:20.789012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d6042-1e63-4b05-8f6a-2c9a2e612300 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:20.789536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d6042-1e63-4b05-8f6a-2c9a2e612300 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:20.789695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d6042-1e63-4b05-8f6a-2c9a2e612300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:20.811377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-eb5d6042-1e63-4b05-8f6a-2c9a2e612300 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:20.812651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-eb5d6042-1e63-4b05-8f6a-2c9a2e612300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:20.886210 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1227/2458] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:30:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:20.898977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863c53c-2f39-4bef-bbef-e5231fb9f7e9 service placement] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:20.907812 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7863c53c-2f39-4bef-bbef-e5231fb9f7e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:20.947546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:30:20.977356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863c53c-2f39-4bef-bbef-e5231fb9f7e9 service placement] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:20.983114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6f06a-9818-4946-acbb-4949cb7bcd9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:20.983710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6f06a-9818-4946-acbb-4949cb7bcd9c service swift] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:20.983987 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6f06a-9818-4946-acbb-4949cb7bcd9c service swift] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:20.984284 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6f06a-9818-4946-acbb-4949cb7bcd9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:20.992459 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6f06a-9818-4946-acbb-4949cb7bcd9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:20.994068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6f06a-9818-4946-acbb-4949cb7bcd9c service swift] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:21.002647 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1232/2459] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:30:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:30:26.612277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d6042-1e63-4b05-8f6a-2c9a2e612300 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:26.614701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-eb5d6042-1e63-4b05-8f6a-2c9a2e612300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:26.631797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d6042-1e63-4b05-8f6a-2c9a2e612300 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:26.638486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcbd67-cac6-41fa-abf2-2a80203073ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:26.639093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcbd67-cac6-41fa-abf2-2a80203073ac service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:26.639373 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcbd67-cac6-41fa-abf2-2a80203073ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:26.639840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcbd67-cac6-41fa-abf2-2a80203073ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:26.641704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-cfdcbd67-cac6-41fa-abf2-2a80203073ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:26.723119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcbd67-cac6-41fa-abf2-2a80203073ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:26.724618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcbd67-cac6-41fa-abf2-2a80203073ac service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:26.736218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1228/2460] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:30.192957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b2e72-abb6-4a45-89e9-7091834c08bd None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:30.193545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b2e72-abb6-4a45-89e9-7091834c08bd None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:30.193545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b2e72-abb6-4a45-89e9-7091834c08bd None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:30.194838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1233/2461] 10.222.0.85 () {54 vars in 921 bytes} [Mon Aug 29 20:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:30:30.201221 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c59b5-a113-49e9-8532-974d7a13dbe6 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:30.201779 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c59b5-a113-49e9-8532-974d7a13dbe6 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:30.201779 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c59b5-a113-49e9-8532-974d7a13dbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:30.223275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-a43c59b5-a113-49e9-8532-974d7a13dbe6 None None] MFA Rules not processed for user `805df3047fee45c7a702d72a5c082eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:30.225543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a43c59b5-a113-49e9-8532-974d7a13dbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:30.281660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:30:30.312240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1229/2462] 10.222.0.85 () {58 vars in 1018 bytes} [Mon Aug 29 20:30:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:30.327270 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b2e72-abb6-4a45-89e9-7091834c08bd None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:30.329142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-180b2e72-abb6-4a45-89e9-7091834c08bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:30.357428 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b2e72-abb6-4a45-89e9-7091834c08bd None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:30.366145 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e119cd-b50c-4358-9fe6-ac47021e58dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:30.366990 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e119cd-b50c-4358-9fe6-ac47021e58dd service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:30.367426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e119cd-b50c-4358-9fe6-ac47021e58dd service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:30.367874 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e119cd-b50c-4358-9fe6-ac47021e58dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:30.382875 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e119cd-b50c-4358-9fe6-ac47021e58dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:30.388257 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e119cd-b50c-4358-9fe6-ac47021e58dd service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:30.401601 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1234/2463] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:40.395468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=22 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:30:40.903084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=11 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:30:43.575409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c5a87-7d45-43b7-9145-31b449ac971f None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:43.575982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c5a87-7d45-43b7-9145-31b449ac971f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:43.575982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c5a87-7d45-43b7-9145-31b449ac971f None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:43.579216 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1230/2464] 10.222.0.85 () {54 vars in 921 bytes} [Mon Aug 29 20:30:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:30:43.585820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502c7cb-1fe8-488c-9b76-58951585b266 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:43.586684 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502c7cb-1fe8-488c-9b76-58951585b266 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:43.586918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502c7cb-1fe8-488c-9b76-58951585b266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:43.607708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-2502c7cb-1fe8-488c-9b76-58951585b266 None None] MFA Rules not processed for user `805df3047fee45c7a702d72a5c082eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:43.608979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2502c7cb-1fe8-488c-9b76-58951585b266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:43.632401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1235/2465] 10.222.0.85 () {58 vars in 1018 bytes} [Mon Aug 29 20:30:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:43.634348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c5a87-7d45-43b7-9145-31b449ac971f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:43.634348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-523c5a87-7d45-43b7-9145-31b449ac971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:43.699592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c5a87-7d45-43b7-9145-31b449ac971f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:43.703064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491c16e-1723-4843-88a2-60c4cf9e7da5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7e768fdd1054e40b6b4910e39a6e533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:43.703664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491c16e-1723-4843-88a2-60c4cf9e7da5 service glance] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:43.703946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491c16e-1723-4843-88a2-60c4cf9e7da5 service glance] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:43.704244 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491c16e-1723-4843-88a2-60c4cf9e7da5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:43.714084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491c16e-1723-4843-88a2-60c4cf9e7da5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:43.715504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491c16e-1723-4843-88a2-60c4cf9e7da5 service glance] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:43.730088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1231/2466] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:44.048267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99016f9e-2db6-4a58-8207-330ebdc61320 None None] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:44.048476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99016f9e-2db6-4a58-8207-330ebdc61320 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:44.048740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99016f9e-2db6-4a58-8207-330ebdc61320 None None] PATH_INFO: `/` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:44.050176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1236/2467] 10.222.0.85 () {54 vars in 921 bytes} [Mon Aug 29 20:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:30:44.054984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f671eef-3a5c-4146-86c8-5b9d50cab486 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:44.055280 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f671eef-3a5c-4146-86c8-5b9d50cab486 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:44.055615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f671eef-3a5c-4146-86c8-5b9d50cab486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:44.079762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-6f671eef-3a5c-4146-86c8-5b9d50cab486 None None] MFA Rules not processed for user `aa222b1c89dc4d2cb1686b4838c7908b`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:44.081059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-6f671eef-3a5c-4146-86c8-5b9d50cab486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:44.088557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1232/2468] 10.222.0.85 () {58 vars in 1018 bytes} [Mon Aug 29 20:30:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:44.102446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99016f9e-2db6-4a58-8207-330ebdc61320 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:44.114248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99016f9e-2db6-4a58-8207-330ebdc61320 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:44.117990 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ede6b-8865-44f7-a155-2c5b41cb4b11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:44.118805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ede6b-8865-44f7-a155-2c5b41cb4b11 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:44.119082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ede6b-8865-44f7-a155-2c5b41cb4b11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:44.119361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ede6b-8865-44f7-a155-2c5b41cb4b11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:44.131521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ede6b-8865-44f7-a155-2c5b41cb4b11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:44.133041 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ede6b-8865-44f7-a155-2c5b41cb4b11 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:44.142877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1237/2469] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:44.240428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b8e65-cb4f-4371-98fa-93b0b7d58b08 None None] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:44.240428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b8e65-cb4f-4371-98fa-93b0b7d58b08 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:44.240428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b8e65-cb4f-4371-98fa-93b0b7d58b08 None None] PATH_INFO: `/` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:44.240428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1233/2470] 10.222.0.85 () {54 vars in 921 bytes} [Mon Aug 29 20:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:30:44.245004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb21317-a764-422c-b759-4090d7ba87f3 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:44.245373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb21317-a764-422c-b759-4090d7ba87f3 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:44.245713 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb21317-a764-422c-b759-4090d7ba87f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:44.266521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-fdb21317-a764-422c-b759-4090d7ba87f3 None None] MFA Rules not processed for user `805df3047fee45c7a702d72a5c082eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:44.267738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-fdb21317-a764-422c-b759-4090d7ba87f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:44.278359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1238/2471] 10.222.0.85 () {58 vars in 1018 bytes} [Mon Aug 29 20:30:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:44.290799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b8e65-cb4f-4371-98fa-93b0b7d58b08 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:44.301777 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b8e65-cb4f-4371-98fa-93b0b7d58b08 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:44.305296 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be26d4-2f07-4c1e-80af-d45c9dbbd27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:44.305884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be26d4-2f07-4c1e-80af-d45c9dbbd27d service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:44.306165 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be26d4-2f07-4c1e-80af-d45c9dbbd27d service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:44.306466 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be26d4-2f07-4c1e-80af-d45c9dbbd27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:44.316360 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be26d4-2f07-4c1e-80af-d45c9dbbd27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:44.317858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be26d4-2f07-4c1e-80af-d45c9dbbd27d service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:44.328765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1234/2472] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:45.756189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb21317-a764-422c-b759-4090d7ba87f3 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:45.766013 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb21317-a764-422c-b759-4090d7ba87f3 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:45.769623 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf6750-f013-441e-ae7e-586a98f1a152 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:45.770700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf6750-f013-441e-ae7e-586a98f1a152 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:45.770815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf6750-f013-441e-ae7e-586a98f1a152 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:45.771062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf6750-f013-441e-ae7e-586a98f1a152 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:45.773035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-19bf6750-f013-441e-ae7e-586a98f1a152 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:45.848909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf6750-f013-441e-ae7e-586a98f1a152 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:45.850509 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf6750-f013-441e-ae7e-586a98f1a152 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:45.862099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1239/2473] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:50.565997 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96380451-c3cf-41c9-8504-51e619c2e169 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:50.567213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96380451-c3cf-41c9-8504-51e619c2e169 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:50.567213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96380451-c3cf-41c9-8504-51e619c2e169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:50.587652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-96380451-c3cf-41c9-8504-51e619c2e169 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:50.588910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-96380451-c3cf-41c9-8504-51e619c2e169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:50.596258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1235/2474] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:30:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:50.636013 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf6750-f013-441e-ae7e-586a98f1a152 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:50.647569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf6750-f013-441e-ae7e-586a98f1a152 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:50.653588 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9f5d8-6783-41a1-9e32-e72b0341ed4e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:50.654096 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9f5d8-6783-41a1-9e32-e72b0341ed4e service swift] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:50.654380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9f5d8-6783-41a1-9e32-e72b0341ed4e service swift] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:50.654654 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9f5d8-6783-41a1-9e32-e72b0341ed4e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:50.663378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9f5d8-6783-41a1-9e32-e72b0341ed4e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:50.665524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9f5d8-6783-41a1-9e32-e72b0341ed4e service swift] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:50.674725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1240/2475] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:30:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:31:00.581751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=38 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:31:00.670564 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=48 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:31:23.368319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b8aaa-4f1c-41cf-8ec9-73059357b7b2 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:23.368791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b8aaa-4f1c-41cf-8ec9-73059357b7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:23.368884 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b8aaa-4f1c-41cf-8ec9-73059357b7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:23.387601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-034b8aaa-4f1c-41cf-8ec9-73059357b7b2 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:31:23.388804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-034b8aaa-4f1c-41cf-8ec9-73059357b7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:31:23.396456 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1236/2476] 10.222.0.85 () {58 vars in 967 bytes} [Mon Aug 29 20:31:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:31:23.410577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9f5d8-6783-41a1-9e32-e72b0341ed4e service swift] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:31:23.422641 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9f5d8-6783-41a1-9e32-e72b0341ed4e service swift] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:31:23.427729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504a19f-536e-41cb-bc46-96311fa2b5d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:31:23.428377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504a19f-536e-41cb-bc46-96311fa2b5d9 service swift] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:23.428660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504a19f-536e-41cb-bc46-96311fa2b5d9 service swift] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:23.428937 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504a19f-536e-41cb-bc46-96311fa2b5d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:23.436961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504a19f-536e-41cb-bc46-96311fa2b5d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:31:23.438613 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504a19f-536e-41cb-bc46-96311fa2b5d9 service swift] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:31:23.446624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1241/2477] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:31:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:31:33.383060 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:31:33.445304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=6 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:31:39.324186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4388844-50a3-4361-a9b6-7874a16b06bc None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:39.324696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4388844-50a3-4361-a9b6-7874a16b06bc None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:39.324810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4388844-50a3-4361-a9b6-7874a16b06bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:39.343976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-c4388844-50a3-4361-a9b6-7874a16b06bc None None] MFA Rules not processed for user `336345e5437341ba9859d776c44ea4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:31:39.345295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c4388844-50a3-4361-a9b6-7874a16b06bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:31:39.402694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1237/2478] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:31:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:31:49.395345 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=25 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:31:57.347297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6dbfaf-187a-430c-b4ee-3e57c79f2585 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:57.347840 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6dbfaf-187a-430c-b4ee-3e57c79f2585 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:57.347840 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6dbfaf-187a-430c-b4ee-3e57c79f2585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:57.367785 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-eb6dbfaf-187a-430c-b4ee-3e57c79f2585 None None] MFA Rules not processed for user `83efc8c932d04432ad3d0c974e0333a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:31:57.369022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-eb6dbfaf-187a-430c-b4ee-3e57c79f2585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:31:57.375743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1242/2479] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:31:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:31:57.411548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4388844-50a3-4361-a9b6-7874a16b06bc None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:31:57.421626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4388844-50a3-4361-a9b6-7874a16b06bc None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:31:57.425352 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f97d2-8a4b-4913-baa9-b098834d971c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7e768fdd1054e40b6b4910e39a6e533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:31:57.425982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f97d2-8a4b-4913-baa9-b098834d971c service glance] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:57.426265 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f97d2-8a4b-4913-baa9-b098834d971c service glance] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:57.426541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f97d2-8a4b-4913-baa9-b098834d971c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:57.428368 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ac3f97d2-8a4b-4913-baa9-b098834d971c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:31:57.441571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f97d2-8a4b-4913-baa9-b098834d971c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:31:57.443082 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f97d2-8a4b-4913-baa9-b098834d971c service glance] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:31:57.454420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1238/2480] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:31:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:31:57.670171 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6dbfaf-187a-430c-b4ee-3e57c79f2585 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:31:57.679825 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6dbfaf-187a-430c-b4ee-3e57c79f2585 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:31:57.683316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61125278-9ad7-4e2a-8bae-779958231eb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:31:57.683914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61125278-9ad7-4e2a-8bae-779958231eb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:57.684224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61125278-9ad7-4e2a-8bae-779958231eb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:57.684500 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61125278-9ad7-4e2a-8bae-779958231eb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:57.686298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-61125278-9ad7-4e2a-8bae-779958231eb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:31:57.700190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61125278-9ad7-4e2a-8bae-779958231eb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:31:57.701625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61125278-9ad7-4e2a-8bae-779958231eb6 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:31:57.712741 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1243/2481] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:31:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:32:07.449653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=6 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:07.707833 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=9 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:11.389384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f97d2-8a4b-4913-baa9-b098834d971c service glance] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:11.390926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ac3f97d2-8a4b-4913-baa9-b098834d971c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:11.408427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f97d2-8a4b-4913-baa9-b098834d971c service glance] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:11.414705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c1e89-4524-4d8c-a08a-8e554de8f934 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:11.416076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c1e89-4524-4d8c-a08a-8e554de8f934 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:11.416076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c1e89-4524-4d8c-a08a-8e554de8f934 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:11.416076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c1e89-4524-4d8c-a08a-8e554de8f934 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:11.417669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2f5c1e89-4524-4d8c-a08a-8e554de8f934 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:11.479486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c1e89-4524-4d8c-a08a-8e554de8f934 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:11.481122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c1e89-4524-4d8c-a08a-8e554de8f934 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:11.490320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1239/2482] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:19.048342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7388b-ecd9-477d-8f66-bf14ba5cc424 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:19.048850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7388b-ecd9-477d-8f66-bf14ba5cc424 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:19.048963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7388b-ecd9-477d-8f66-bf14ba5cc424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:19.068943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-0eb7388b-ecd9-477d-8f66-bf14ba5cc424 None None] MFA Rules not processed for user `3f4b05a527e14d2599e03077c6c5f96f`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:19.070203 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-0eb7388b-ecd9-477d-8f66-bf14ba5cc424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:19.077798 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1244/2483] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:32:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:19.394219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c1e89-4524-4d8c-a08a-8e554de8f934 service nova] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:19.405289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c1e89-4524-4d8c-a08a-8e554de8f934 service nova] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:19.413094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424f542-e889-4666-97c8-bc4566d8b28f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:19.413094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424f542-e889-4666-97c8-bc4566d8b28f service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:19.413094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424f542-e889-4666-97c8-bc4566d8b28f service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:19.413094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424f542-e889-4666-97c8-bc4566d8b28f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:19.413553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a424f542-e889-4666-97c8-bc4566d8b28f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:19.422732 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424f542-e889-4666-97c8-bc4566d8b28f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:19.424338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424f542-e889-4666-97c8-bc4566d8b28f service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:19.433260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1240/2484] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:19.941843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7388b-ecd9-477d-8f66-bf14ba5cc424 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:19.954185 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7388b-ecd9-477d-8f66-bf14ba5cc424 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:19.959223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bc94c-b495-46ff-8cbb-c57821244963 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:19.959827 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bc94c-b495-46ff-8cbb-c57821244963 service placement] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:19.960129 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bc94c-b495-46ff-8cbb-c57821244963 service placement] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:19.960430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bc94c-b495-46ff-8cbb-c57821244963 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:19.962128 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-118bc94c-b495-46ff-8cbb-c57821244963 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:19.974589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bc94c-b495-46ff-8cbb-c57821244963 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:19.978559 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bc94c-b495-46ff-8cbb-c57821244963 service placement] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:20.001277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1245/2485] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:29.429446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=34 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:29.982645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=9 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:30.447935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424f542-e889-4666-97c8-bc4566d8b28f service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:30.456883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424f542-e889-4666-97c8-bc4566d8b28f service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:30.461204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2d9d0-8418-4c7d-b583-3935e55e26dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:30.461959 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2d9d0-8418-4c7d-b583-3935e55e26dd service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:30.462260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2d9d0-8418-4c7d-b583-3935e55e26dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:30.462533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2d9d0-8418-4c7d-b583-3935e55e26dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:30.464281 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a5c2d9d0-8418-4c7d-b583-3935e55e26dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:30.473883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2d9d0-8418-4c7d-b583-3935e55e26dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:30.475967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2d9d0-8418-4c7d-b583-3935e55e26dd service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:30.484033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1241/2486] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:31.447936 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bc94c-b495-46ff-8cbb-c57821244963 service placement] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:31.460978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bc94c-b495-46ff-8cbb-c57821244963 service placement] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:31.466630 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95534aeb-1d83-45f0-8c0b-db6e0fbd1918 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:31.467703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95534aeb-1d83-45f0-8c0b-db6e0fbd1918 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:31.467703 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95534aeb-1d83-45f0-8c0b-db6e0fbd1918 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:31.468076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95534aeb-1d83-45f0-8c0b-db6e0fbd1918 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:31.474240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-95534aeb-1d83-45f0-8c0b-db6e0fbd1918 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:31.483410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95534aeb-1d83-45f0-8c0b-db6e0fbd1918 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:31.485063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95534aeb-1d83-45f0-8c0b-db6e0fbd1918 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:31.496319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1246/2487] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:34.172768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2d9d0-8418-4c7d-b583-3935e55e26dd service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:34.174366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a5c2d9d0-8418-4c7d-b583-3935e55e26dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:34.186459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2d9d0-8418-4c7d-b583-3935e55e26dd service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:34.190113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d08a4-b3b6-44de-8922-8cde3c8d1d41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:34.190711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d08a4-b3b6-44de-8922-8cde3c8d1d41 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:34.191053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d08a4-b3b6-44de-8922-8cde3c8d1d41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:34.191335 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d08a4-b3b6-44de-8922-8cde3c8d1d41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:34.193169 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b58d08a4-b3b6-44de-8922-8cde3c8d1d41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:34.204869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d08a4-b3b6-44de-8922-8cde3c8d1d41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:34.206366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d08a4-b3b6-44de-8922-8cde3c8d1d41 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:34.216570 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1242/2488] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:34 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:41.491496 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=6 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:42.572280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfd874-5f25-47c7-a913-a0b0b4294382 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:42.572856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfd874-5f25-47c7-a913-a0b0b4294382 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:42.572856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfd874-5f25-47c7-a913-a0b0b4294382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:42.592322 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-82cfd874-5f25-47c7-a913-a0b0b4294382 None None] MFA Rules not processed for user `859d36aa86bd403ca1d46b245efaef61`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:42.593548 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-82cfd874-5f25-47c7-a913-a0b0b4294382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:42.650006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1247/2489] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:32:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:44.211949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=12 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:52.642026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=25 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:00.667287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d08a4-b3b6-44de-8922-8cde3c8d1d41 service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:00.681464 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d08a4-b3b6-44de-8922-8cde3c8d1d41 service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:00.687614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d6721-1ce1-4211-b6fc-59f1671fc6ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:00.688250 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d6721-1ce1-4211-b6fc-59f1671fc6ee service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:00.689252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d6721-1ce1-4211-b6fc-59f1671fc6ee service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:00.689533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d6721-1ce1-4211-b6fc-59f1671fc6ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:00.691270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ce0d6721-1ce1-4211-b6fc-59f1671fc6ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:00.704752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d6721-1ce1-4211-b6fc-59f1671fc6ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:00.706346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d6721-1ce1-4211-b6fc-59f1671fc6ee service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:00.718541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1243/2490] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:10.714540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=6 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:13.132527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfd874-5f25-47c7-a913-a0b0b4294382 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:13.142295 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfd874-5f25-47c7-a913-a0b0b4294382 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:13.146497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a4b44-74f6-43c4-bbcb-b0299a659dc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:13.147388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a4b44-74f6-43c4-bbcb-b0299a659dc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:13.147670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a4b44-74f6-43c4-bbcb-b0299a659dc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:13.147945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a4b44-74f6-43c4-bbcb-b0299a659dc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:13.149758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8a7a4b44-74f6-43c4-bbcb-b0299a659dc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:13.162604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a4b44-74f6-43c4-bbcb-b0299a659dc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:13.164045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a4b44-74f6-43c4-bbcb-b0299a659dc1 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:13.174169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1248/2491] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:17.624660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d6721-1ce1-4211-b6fc-59f1671fc6ee service nova] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:17.626647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ce0d6721-1ce1-4211-b6fc-59f1671fc6ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:17.643994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d6721-1ce1-4211-b6fc-59f1671fc6ee service nova] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:17.651400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f46f1d-4cdb-4674-a4f3-c1d78545361e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:17.652403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f46f1d-4cdb-4674-a4f3-c1d78545361e service placement] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:17.652793 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f46f1d-4cdb-4674-a4f3-c1d78545361e service placement] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:17.653343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f46f1d-4cdb-4674-a4f3-c1d78545361e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:17.655395 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-90f46f1d-4cdb-4674-a4f3-c1d78545361e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:17.668783 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f46f1d-4cdb-4674-a4f3-c1d78545361e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:17.670509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f46f1d-4cdb-4674-a4f3-c1d78545361e service placement] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:17.682087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1244/2492] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:18.089685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a4b44-74f6-43c4-bbcb-b0299a659dc1 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:18.092477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8a7a4b44-74f6-43c4-bbcb-b0299a659dc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:18.105918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a4b44-74f6-43c4-bbcb-b0299a659dc1 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:18.115939 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:18.116515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:18.117007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:18.118256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] PATH_INFO: `/v3/users/336345e5437341ba9859d776c44ea4be` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.120133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=336345e5437341ba9859d776c44ea4be)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:18.122191 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:18.495481 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336345e5437341ba9859d776c44ea4be'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:18.503743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336345e5437341ba9859d776c44ea4be'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:18.632734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336345e5437341ba9859d776c44ea4be'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:18.633450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336345e5437341ba9859d776c44ea4be'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:18.649666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336345e5437341ba9859d776c44ea4be'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:18.662300 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1249/2493] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:18 2022] DELETE /identity/v3/users/336345e5437341ba9859d776c44ea4be => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:18.866921 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f46f1d-4cdb-4674-a4f3-c1d78545361e service placement] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:18.876608 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f46f1d-4cdb-4674-a4f3-c1d78545361e service placement] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:18.938327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496e53c-ae18-4175-8e81-002d5a7e1387 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:18.938943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496e53c-ae18-4175-8e81-002d5a7e1387 service placement] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:18.939224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496e53c-ae18-4175-8e81-002d5a7e1387 service placement] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:18.939663 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496e53c-ae18-4175-8e81-002d5a7e1387 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.941428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7496e53c-ae18-4175-8e81-002d5a7e1387 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:18.952973 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496e53c-ae18-4175-8e81-002d5a7e1387 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:18.954552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496e53c-ae18-4175-8e81-002d5a7e1387 service placement] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:18.965174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1245/2494] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:19.007049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:19.027305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:19.081582 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496e53c-ae18-4175-8e81-002d5a7e1387 service placement] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:19.084289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7496e53c-ae18-4175-8e81-002d5a7e1387 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:19.101652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c81018-1c0d-4ebc-8fde-2b2c3834da4f admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:19.102433 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496e53c-ae18-4175-8e81-002d5a7e1387 service placement] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:19.106327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b0432-af54-4d8a-88e6-a06f5675ec68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:19.106963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b0432-af54-4d8a-88e6-a06f5675ec68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:19.107094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892c9b7-c108-4b7e-baa1-c7a99fad5e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:19.107241 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b0432-af54-4d8a-88e6-a06f5675ec68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:19.107751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892c9b7-c108-4b7e-baa1-c7a99fad5e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:19.107875 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b0432-af54-4d8a-88e6-a06f5675ec68 admin admin] PATH_INFO: `/v3/projects/66be3e70cbf742269fe9306a7b7e8fe0` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:19.108033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892c9b7-c108-4b7e-baa1-c7a99fad5e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:19.108523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892c9b7-c108-4b7e-baa1-c7a99fad5e27 admin admin] PATH_INFO: `/v3/users/83efc8c932d04432ad3d0c974e0333a9` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:19.109091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b0432-af54-4d8a-88e6-a06f5675ec68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66be3e70cbf742269fe9306a7b7e8fe0)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:19.109782 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892c9b7-c108-4b7e-baa1-c7a99fad5e27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83efc8c932d04432ad3d0c974e0333a9)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:19.110992 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b0432-af54-4d8a-88e6-a06f5675ec68 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:19.111722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892c9b7-c108-4b7e-baa1-c7a99fad5e27 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:19.465062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b77b0432-af54-4d8a-88e6-a06f5675ec68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66be3e70cbf742269fe9306a7b7e8fe0', 'request_id': 'req-b77b0432-af54-4d8a-88e6-a06f5675ec68'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.465271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1892c9b7-c108-4b7e-baa1-c7a99fad5e27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83efc8c932d04432ad3d0c974e0333a9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.465847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1892c9b7-c108-4b7e-baa1-c7a99fad5e27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83efc8c932d04432ad3d0c974e0333a9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.472117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b77b0432-af54-4d8a-88e6-a06f5675ec68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66be3e70cbf742269fe9306a7b7e8fe0', 'request_id': 'req-b77b0432-af54-4d8a-88e6-a06f5675ec68'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.473337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1892c9b7-c108-4b7e-baa1-c7a99fad5e27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83efc8c932d04432ad3d0c974e0333a9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.477060 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:19.478319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b77b0432-af54-4d8a-88e6-a06f5675ec68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66be3e70cbf742269fe9306a7b7e8fe0', 'request_id': 'req-b77b0432-af54-4d8a-88e6-a06f5675ec68'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.678918 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1892c9b7-c108-4b7e-baa1-c7a99fad5e27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83efc8c932d04432ad3d0c974e0333a9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.679710 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-1892c9b7-c108-4b7e-baa1-c7a99fad5e27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83efc8c932d04432ad3d0c974e0333a9'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.698167 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1246/2495] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:19 2022] DELETE /identity/v3/users/83efc8c932d04432ad3d0c974e0333a9 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:19.877701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b77b0432-af54-4d8a-88e6-a06f5675ec68 admin admin] The token cache is being invalidate because project 66be3e70cbf742269fe9306a7b7e8fe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:19.878029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b77b0432-af54-4d8a-88e6-a06f5675ec68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.878667 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b77b0432-af54-4d8a-88e6-a06f5675ec68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.880919 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1250/2496] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:19 2022] DELETE /identity/v3/projects/66be3e70cbf742269fe9306a7b7e8fe0 => generated 0 bytes in 875 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:19.885936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824e6a9-ea18-4564-ad72-c775a6781f37 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:19.886474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824e6a9-ea18-4564-ad72-c775a6781f37 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:19.886630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824e6a9-ea18-4564-ad72-c775a6781f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:19.907586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-5824e6a9-ea18-4564-ad72-c775a6781f37 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:19.908861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5824e6a9-ea18-4564-ad72-c775a6781f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:19.954216 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:19.969928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1247/2497] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:19.978136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00fd48-b462-4b95-ac90-dd3d3316fadd None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:19.978443 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00fd48-b462-4b95-ac90-dd3d3316fadd None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:19.978740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00fd48-b462-4b95-ac90-dd3d3316fadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.000256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ce00fd48-b462-4b95-ac90-dd3d3316fadd None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:20.001165 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824e6a9-ea18-4564-ad72-c775a6781f37 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:20.001929 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ce00fd48-b462-4b95-ac90-dd3d3316fadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:20.002726 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5824e6a9-ea18-4564-ad72-c775a6781f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:20.016458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1251/2498] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:20.017660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00fd48-b462-4b95-ac90-dd3d3316fadd None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:20.017801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824e6a9-ea18-4564-ad72-c775a6781f37 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:20.026077 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:20.031554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.031554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.031554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] PATH_INFO: `/v3/projects/ac65a0d70d7045d183a8a725b304c169` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.031554 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac65a0d70d7045d183a8a725b304c169)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:20.032442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00fd48-b462-4b95-ac90-dd3d3316fadd None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:20.032707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:20.035573 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3c829-946d-494d-ac05-2bfbb4a8f615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:20.036192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3c829-946d-494d-ac05-2bfbb4a8f615 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.036479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3c829-946d-494d-ac05-2bfbb4a8f615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.036745 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3c829-946d-494d-ac05-2bfbb4a8f615 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.037391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf3c829-946d-494d-ac05-2bfbb4a8f615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:20.037713 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf3c829-946d-494d-ac05-2bfbb4a8f615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:20.039459 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf3c829-946d-494d-ac05-2bfbb4a8f615 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:20.052151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1252/2499] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:33:20.056457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3c829-946d-494d-ac05-2bfbb4a8f615 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:20.066246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3c829-946d-494d-ac05-2bfbb4a8f615 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:20.071424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063b8de-adbb-4784-8ad6-bcc4ff8423f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:20.077499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063b8de-adbb-4784-8ad6-bcc4ff8423f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.077499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063b8de-adbb-4784-8ad6-bcc4ff8423f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.077499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063b8de-adbb-4784-8ad6-bcc4ff8423f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.077499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063b8de-adbb-4784-8ad6-bcc4ff8423f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:20.077499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063b8de-adbb-4784-8ad6-bcc4ff8423f5 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:20.159426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac65a0d70d7045d183a8a725b304c169', 'request_id': 'req-5e8f2030-7559-43e4-ab79-598f3b29adef'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:20.165098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac65a0d70d7045d183a8a725b304c169', 'request_id': 'req-5e8f2030-7559-43e4-ab79-598f3b29adef'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:20.188316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1253/2500] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:20 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:20.193253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063b8de-adbb-4784-8ad6-bcc4ff8423f5 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:20.259692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063b8de-adbb-4784-8ad6-bcc4ff8423f5 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:20.266820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c4ee7-fc14-4924-811b-ee3239bdb10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:20.267591 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c4ee7-fc14-4924-811b-ee3239bdb10c admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.267999 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c4ee7-fc14-4924-811b-ee3239bdb10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.268410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c4ee7-fc14-4924-811b-ee3239bdb10c admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.269167 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c4ee7-fc14-4924-811b-ee3239bdb10c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:20.272547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c4ee7-fc14-4924-811b-ee3239bdb10c admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:20.286564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac65a0d70d7045d183a8a725b304c169', 'request_id': 'req-5e8f2030-7559-43e4-ab79-598f3b29adef'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:20.302841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] The token cache is being invalidate because project ac65a0d70d7045d183a8a725b304c169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:20.303572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:20.304382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:20.307239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1248/2501] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:19 2022] DELETE /identity/v3/projects/ac65a0d70d7045d183a8a725b304c169 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:20.443739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1254/2502] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:20 2022] POST /identity/v3/users => generated 292 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:20.448040 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:20.449560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:20.538280 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f2030-7559-43e4-ab79-598f3b29adef admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:20.543185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107c658-98b5-40de-aa20-8efae570bd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:20.543790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107c658-98b5-40de-aa20-8efae570bd3e admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.544068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107c658-98b5-40de-aa20-8efae570bd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.544383 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107c658-98b5-40de-aa20-8efae570bd3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.545127 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107c658-98b5-40de-aa20-8efae570bd3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:20.546654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107c658-98b5-40de-aa20-8efae570bd3e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:20.560869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1249/2503] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:20.561932 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c4ee7-fc14-4924-811b-ee3239bdb10c admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:20.564252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcad30b-8221-4a6e-9b4c-ffaef132678e None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.565413 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcad30b-8221-4a6e-9b4c-ffaef132678e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.565697 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcad30b-8221-4a6e-9b4c-ffaef132678e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.574495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c4ee7-fc14-4924-811b-ee3239bdb10c admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:20.580770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc91e7-d3e2-4204-a17c-8bb8b9c592c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:20.581383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc91e7-d3e2-4204-a17c-8bb8b9c592c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.581663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc91e7-d3e2-4204-a17c-8bb8b9c592c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.581940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc91e7-d3e2-4204-a17c-8bb8b9c592c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.582662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc91e7-d3e2-4204-a17c-8bb8b9c592c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:20.584328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc91e7-d3e2-4204-a17c-8bb8b9c592c1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:20.591564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-0bcad30b-8221-4a6e-9b4c-ffaef132678e None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:20.592807 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0bcad30b-8221-4a6e-9b4c-ffaef132678e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:20.595375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1255/2504] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:20.600363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc91e7-d3e2-4204-a17c-8bb8b9c592c1 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:20.611336 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc91e7-d3e2-4204-a17c-8bb8b9c592c1 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:20.617530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a40c2-355e-4001-b518-05e73dec11cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:20.617654 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a40c2-355e-4001-b518-05e73dec11cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.617944 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a40c2-355e-4001-b518-05e73dec11cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.618221 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a40c2-355e-4001-b518-05e73dec11cd admin admin] PATH_INFO: `/v3/projects/f7b499be5a424f8a860c6e30a060d941/users/75a2f770cc774d61baa6c07532665477/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.634260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a40c2-355e-4001-b518-05e73dec11cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b499be5a424f8a860c6e30a060d941, user_id=75a2f770cc774d61baa6c07532665477, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:20.637290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a40c2-355e-4001-b518-05e73dec11cd admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:20.660460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1250/2505] 10.222.0.85 () {58 vars in 967 bytes} [Mon Aug 29 20:33:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:20.793593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1256/2506] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:20 2022] PUT /identity/v3/projects/f7b499be5a424f8a860c6e30a060d941/users/75a2f770cc774d61baa6c07532665477/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:20.798812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69536a80-50a9-4da6-a293-53cc3d0328f3 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.799101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69536a80-50a9-4da6-a293-53cc3d0328f3 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.799380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69536a80-50a9-4da6-a293-53cc3d0328f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.818623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-69536a80-50a9-4da6-a293-53cc3d0328f3 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:20.819893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-69536a80-50a9-4da6-a293-53cc3d0328f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:20.871849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a40c2-355e-4001-b518-05e73dec11cd admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:20.873644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-0b7a40c2-355e-4001-b518-05e73dec11cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:20.878564 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1251/2507] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:20.888429 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69536a80-50a9-4da6-a293-53cc3d0328f3 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:20.889955 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-69536a80-50a9-4da6-a293-53cc3d0328f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:20.945585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:20.950621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a40c2-355e-4001-b518-05e73dec11cd admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:20.956318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f0e8e-8356-4416-ad38-72d216da1dac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:20.956995 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f0e8e-8356-4416-ad38-72d216da1dac service swift] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.957377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f0e8e-8356-4416-ad38-72d216da1dac service swift] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.957653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f0e8e-8356-4416-ad38-72d216da1dac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.958401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69536a80-50a9-4da6-a293-53cc3d0328f3 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:20.962088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b5256-10f1-43bd-a7b5-c7d6a56019f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:20.962754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b5256-10f1-43bd-a7b5-c7d6a56019f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.963073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b5256-10f1-43bd-a7b5-c7d6a56019f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.963482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b5256-10f1-43bd-a7b5-c7d6a56019f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.973637 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b5256-10f1-43bd-a7b5-c7d6a56019f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:20.975447 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b5256-10f1-43bd-a7b5-c7d6a56019f6 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:20.986623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1252/2508] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:21.012358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:21.025925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f0e8e-8356-4416-ad38-72d216da1dac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:21.027519 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f0e8e-8356-4416-ad38-72d216da1dac service swift] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:21.038362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1257/2509] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:33:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 20:33:29.501227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03399780-e0eb-4b55-8bf1-9304d2ad1478 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:29.501775 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03399780-e0eb-4b55-8bf1-9304d2ad1478 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:29.501775 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03399780-e0eb-4b55-8bf1-9304d2ad1478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:29.531850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-03399780-e0eb-4b55-8bf1-9304d2ad1478 None None] MFA Rules not processed for user `75a2f770cc774d61baa6c07532665477`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:29.533202 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-03399780-e0eb-4b55-8bf1-9304d2ad1478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:29.610313 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1253/2510] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:33:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:29.624115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f0e8e-8356-4416-ad38-72d216da1dac service swift] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:29.625635 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-4f9f0e8e-8356-4416-ad38-72d216da1dac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:29.712563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f0e8e-8356-4416-ad38-72d216da1dac service swift] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:29.718663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afa942-aade-4511-bc24-92964dc57d75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:29.719719 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afa942-aade-4511-bc24-92964dc57d75 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:29.719719 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afa942-aade-4511-bc24-92964dc57d75 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:29.719927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afa942-aade-4511-bc24-92964dc57d75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:29.728597 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afa942-aade-4511-bc24-92964dc57d75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:29.730229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afa942-aade-4511-bc24-92964dc57d75 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:29.739837 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1258/2511] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:33.122000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f6181-9704-4c80-9196-a8a32a1e3a65 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:33.122547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f6181-9704-4c80-9196-a8a32a1e3a65 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:33.122646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f6181-9704-4c80-9196-a8a32a1e3a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:33.142724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-040f6181-9704-4c80-9196-a8a32a1e3a65 None None] MFA Rules not processed for user `75a2f770cc774d61baa6c07532665477`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:33.143927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-040f6181-9704-4c80-9196-a8a32a1e3a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:33.151666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1254/2512] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:33:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:33.263245 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afa942-aade-4511-bc24-92964dc57d75 service nova] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:33.264727 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c2afa942-aade-4511-bc24-92964dc57d75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:33.279332 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afa942-aade-4511-bc24-92964dc57d75 service nova] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:33.284212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:33.284903 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:33.285748 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:33.286091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] PATH_INFO: `/v3/users/e82c9e9f4ca742abb6531b3210be3776` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:33.287390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e82c9e9f4ca742abb6531b3210be3776)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:33.289438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:33.544734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e82c9e9f4ca742abb6531b3210be3776'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:33.553224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e82c9e9f4ca742abb6531b3210be3776'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:33.646007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e82c9e9f4ca742abb6531b3210be3776'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:33.651715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e82c9e9f4ca742abb6531b3210be3776'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:33.669465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e82c9e9f4ca742abb6531b3210be3776'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:33.672538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:33.678775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1259/2513] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:33 2022] DELETE /identity/v3/users/e82c9e9f4ca742abb6531b3210be3776 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:34.013899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f6181-9704-4c80-9196-a8a32a1e3a65 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:34.028120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f6181-9704-4c80-9196-a8a32a1e3a65 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:34.109178 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:34.132065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:34.132725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:34.132988 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:34.133264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] PATH_INFO: `/v3/projects/0931206111ba46198d58746f57266c23` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:34.134414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0931206111ba46198d58746f57266c23)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:34.136330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:34.319750 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0931206111ba46198d58746f57266c23', 'request_id': 'req-57bdedf0-c57a-4f27-a533-ec984d600068'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.327578 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0931206111ba46198d58746f57266c23', 'request_id': 'req-57bdedf0-c57a-4f27-a533-ec984d600068'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.438704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0931206111ba46198d58746f57266c23', 'request_id': 'req-57bdedf0-c57a-4f27-a533-ec984d600068'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.453181 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] The token cache is being invalidate because project 0931206111ba46198d58746f57266c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:34.453708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.454640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.456514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1255/2514] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:34 2022] DELETE /identity/v3/projects/0931206111ba46198d58746f57266c23 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:34.460571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:34.463127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:34.479585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c5281-93f6-4cad-909b-b99bc38270d8 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:34.537020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:34.537646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:34.537931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:34.538207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] PATH_INFO: `/v3/users/0f670b6b79d24a679364ab356fbc9562` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:34.539340 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f670b6b79d24a679364ab356fbc9562)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:34.541308 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:34.846750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f670b6b79d24a679364ab356fbc9562'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.854961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f670b6b79d24a679364ab356fbc9562'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.946933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f670b6b79d24a679364ab356fbc9562'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.947850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f670b6b79d24a679364ab356fbc9562'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.963177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f670b6b79d24a679364ab356fbc9562'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.970196 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1260/2515] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:34 2022] DELETE /identity/v3/users/0f670b6b79d24a679364ab356fbc9562 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:35.240522 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:35.242037 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:35.255302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdedf0-c57a-4f27-a533-ec984d600068 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:35.322679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:35.323307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:35.323589 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:35.323868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] PATH_INFO: `/v3/projects/789d1fa851cf429a8752fd7df78a8932` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:35.325071 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=789d1fa851cf429a8752fd7df78a8932)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:35.326988 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:35.446091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '789d1fa851cf429a8752fd7df78a8932', 'request_id': 'req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:35.451755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '789d1fa851cf429a8752fd7df78a8932', 'request_id': 'req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:35.575658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '789d1fa851cf429a8752fd7df78a8932', 'request_id': 'req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:35.586176 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:35.590217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] The token cache is being invalidate because project 789d1fa851cf429a8752fd7df78a8932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:35.590545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:35.591173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:35.593019 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1256/2516] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:35 2022] DELETE /identity/v3/projects/789d1fa851cf429a8752fd7df78a8932 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:46.121364 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:47.179371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:47.184347 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:47.260338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840862f-5361-46c5-8c6f-33566724e347 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:47.329742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f74dcc-18b8-4856-8836-0fe997de0127 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:47.330517 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f74dcc-18b8-4856-8836-0fe997de0127 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:47.330883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f74dcc-18b8-4856-8836-0fe997de0127 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:47.331233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f74dcc-18b8-4856-8836-0fe997de0127 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:47.334137 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a2f74dcc-18b8-4856-8836-0fe997de0127 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:47.398174 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f74dcc-18b8-4856-8836-0fe997de0127 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:47.399799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f74dcc-18b8-4856-8836-0fe997de0127 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:47.409235 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1261/2517] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:57.404912 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=84 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:13.535046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:13.536484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:13.595856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93dd4f9-ecd9-40bd-8175-3c1f7b701dcd admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:13.600046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6ece0-cdf8-4f1f-910e-e759e70653c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:13.602434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6ece0-cdf8-4f1f-910e-e759e70653c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:13.602434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6ece0-cdf8-4f1f-910e-e759e70653c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:13.602434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6ece0-cdf8-4f1f-910e-e759e70653c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:13.602951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-44a6ece0-cdf8-4f1f-910e-e759e70653c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:13.666793 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6ece0-cdf8-4f1f-910e-e759e70653c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:13.668159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6ece0-cdf8-4f1f-910e-e759e70653c3 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:13.677800 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1257/2518] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:15.723557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ad14f-d96b-4ba0-a67e-cb68f1aad400 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:15.724190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ad14f-d96b-4ba0-a67e-cb68f1aad400 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:15.724190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ad14f-d96b-4ba0-a67e-cb68f1aad400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:15.745828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-153ad14f-d96b-4ba0-a67e-cb68f1aad400 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:15.746826 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-153ad14f-d96b-4ba0-a67e-cb68f1aad400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:15.826889 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1262/2519] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:34:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:15.842165 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6ece0-cdf8-4f1f-910e-e759e70653c3 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:15.844038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-44a6ece0-cdf8-4f1f-910e-e759e70653c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:15.955842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6ece0-cdf8-4f1f-910e-e759e70653c3 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:15.962581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec7d31-5582-4541-8410-eb6f4355de7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:15.963196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec7d31-5582-4541-8410-eb6f4355de7f service swift] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:15.963503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec7d31-5582-4541-8410-eb6f4355de7f service swift] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:15.963780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec7d31-5582-4541-8410-eb6f4355de7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:15.972888 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec7d31-5582-4541-8410-eb6f4355de7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:15.974503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec7d31-5582-4541-8410-eb6f4355de7f service swift] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:15.986290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1258/2520] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:34:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:34:17.912500 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ad14f-d96b-4ba0-a67e-cb68f1aad400 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:17.914291 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-153ad14f-d96b-4ba0-a67e-cb68f1aad400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:17.939333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ad14f-d96b-4ba0-a67e-cb68f1aad400 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:17.946715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf55ce-0ba6-42b5-967f-a7ddbd09f1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:17.947599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf55ce-0ba6-42b5-967f-a7ddbd09f1fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:17.948048 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf55ce-0ba6-42b5-967f-a7ddbd09f1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:17.948356 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf55ce-0ba6-42b5-967f-a7ddbd09f1fd admin admin] PATH_INFO: `/v3/users/3f4b05a527e14d2599e03077c6c5f96f` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:17.949659 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf55ce-0ba6-42b5-967f-a7ddbd09f1fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f4b05a527e14d2599e03077c6c5f96f)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:17.951632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf55ce-0ba6-42b5-967f-a7ddbd09f1fd admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:18.174104 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-faaf55ce-0ba6-42b5-967f-a7ddbd09f1fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f4b05a527e14d2599e03077c6c5f96f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:18.182402 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-faaf55ce-0ba6-42b5-967f-a7ddbd09f1fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f4b05a527e14d2599e03077c6c5f96f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:18.249047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-faaf55ce-0ba6-42b5-967f-a7ddbd09f1fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f4b05a527e14d2599e03077c6c5f96f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:18.249599 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-faaf55ce-0ba6-42b5-967f-a7ddbd09f1fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f4b05a527e14d2599e03077c6c5f96f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:18.264136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-faaf55ce-0ba6-42b5-967f-a7ddbd09f1fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f4b05a527e14d2599e03077c6c5f96f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:18.275169 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1263/2521] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:17 2022] DELETE /identity/v3/users/3f4b05a527e14d2599e03077c6c5f96f => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:18.567550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec7d31-5582-4541-8410-eb6f4355de7f service swift] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:18.592468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec7d31-5582-4541-8410-eb6f4355de7f service swift] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:18.661277 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:18.685830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:18.686764 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:18.687249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:18.687706 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d admin admin] PATH_INFO: `/v3/projects/6adb6c50ed6c4fa5beebf037da1e73e7` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:18.689278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6adb6c50ed6c4fa5beebf037da1e73e7)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:18.692340 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:18.839894 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6adb6c50ed6c4fa5beebf037da1e73e7', 'request_id': 'req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:18.850935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6adb6c50ed6c4fa5beebf037da1e73e7', 'request_id': 'req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:18.946174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6adb6c50ed6c4fa5beebf037da1e73e7', 'request_id': 'req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:18.962065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d admin admin] The token cache is being invalidate because project 6adb6c50ed6c4fa5beebf037da1e73e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:18.962065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:18.962065 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8ed5afc8-64a5-4de5-9066-3ccaffdc536d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:18.963220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1259/2522] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:18 2022] DELETE /identity/v3/projects/6adb6c50ed6c4fa5beebf037da1e73e7 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:18.969691 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cacd47-99c7-4726-8107-20165e5421c6 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:18.970209 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cacd47-99c7-4726-8107-20165e5421c6 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:18.970209 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cacd47-99c7-4726-8107-20165e5421c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:18.997317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-42cacd47-99c7-4726-8107-20165e5421c6 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:18.998869 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-42cacd47-99c7-4726-8107-20165e5421c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:19.082626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1264/2523] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:19.090424 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a5e24-6446-41e1-88c8-960b8ac4f727 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:19.090657 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a5e24-6446-41e1-88c8-960b8ac4f727 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:19.090935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a5e24-6446-41e1-88c8-960b8ac4f727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:19.114324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-ee6a5e24-6446-41e1-88c8-960b8ac4f727 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:19.115632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ee6a5e24-6446-41e1-88c8-960b8ac4f727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:19.124407 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1260/2524] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:19.129234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cacd47-99c7-4726-8107-20165e5421c6 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:19.139650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cacd47-99c7-4726-8107-20165e5421c6 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:19.145430 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b436d9-a792-438b-85f5-f39567f492d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:19.146239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b436d9-a792-438b-85f5-f39567f492d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:19.146617 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b436d9-a792-438b-85f5-f39567f492d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:19.146963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b436d9-a792-438b-85f5-f39567f492d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:19.147982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b436d9-a792-438b-85f5-f39567f492d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:19.148390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b436d9-a792-438b-85f5-f39567f492d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:19.150225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b436d9-a792-438b-85f5-f39567f492d6 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:19.164681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1265/2525] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:19.169574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a5e24-6446-41e1-88c8-960b8ac4f727 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:19.179527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a5e24-6446-41e1-88c8-960b8ac4f727 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:19.184759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ab56a-73c0-4be3-a699-f6312a2d6783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:19.185393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ab56a-73c0-4be3-a699-f6312a2d6783 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:19.185678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ab56a-73c0-4be3-a699-f6312a2d6783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:19.185952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ab56a-73c0-4be3-a699-f6312a2d6783 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:19.186607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ab56a-73c0-4be3-a699-f6312a2d6783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:19.188645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ab56a-73c0-4be3-a699-f6312a2d6783 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:19.765039 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1261/2526] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:19 2022] POST /identity/v3/projects => generated 365 bytes in 596 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:19.769317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b436d9-a792-438b-85f5-f39567f492d6 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:19.775833 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:19.779566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b436d9-a792-438b-85f5-f39567f492d6 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:19.843606 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c880c-e19f-46d4-81e0-cc7b95113af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:19.844243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c880c-e19f-46d4-81e0-cc7b95113af3 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:19.844529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c880c-e19f-46d4-81e0-cc7b95113af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:19.844805 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c880c-e19f-46d4-81e0-cc7b95113af3 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:19.845486 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c880c-e19f-46d4-81e0-cc7b95113af3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:19.847986 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c880c-e19f-46d4-81e0-cc7b95113af3 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:19.993212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1266/2527] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:19 2022] POST /identity/v3/users => generated 283 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:19.998655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ab56a-73c0-4be3-a699-f6312a2d6783 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:20.009175 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ab56a-73c0-4be3-a699-f6312a2d6783 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:20.013877 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8b2ae-d55c-4d06-9a0f-c5a69fa3ad3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:20.014509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8b2ae-d55c-4d06-9a0f-c5a69fa3ad3c admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:20.014792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8b2ae-d55c-4d06-9a0f-c5a69fa3ad3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:20.015084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8b2ae-d55c-4d06-9a0f-c5a69fa3ad3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:20.015713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8b2ae-d55c-4d06-9a0f-c5a69fa3ad3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:20.017377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8b2ae-d55c-4d06-9a0f-c5a69fa3ad3c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:20.028975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1262/2528] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:20.033081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c880c-e19f-46d4-81e0-cc7b95113af3 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:20.043902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c880c-e19f-46d4-81e0-cc7b95113af3 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:20.049602 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f5d4d-2a88-40dc-8260-bc0fec510771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:20.049858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f5d4d-2a88-40dc-8260-bc0fec510771 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:20.050140 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f5d4d-2a88-40dc-8260-bc0fec510771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:20.050415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f5d4d-2a88-40dc-8260-bc0fec510771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:20.051070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f5d4d-2a88-40dc-8260-bc0fec510771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:20.052759 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f5d4d-2a88-40dc-8260-bc0fec510771 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:20.065360 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1267/2529] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:34:20.069585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8b2ae-d55c-4d06-9a0f-c5a69fa3ad3c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:20.082615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8b2ae-d55c-4d06-9a0f-c5a69fa3ad3c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:20.090417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0f3bd-520f-48b6-9e48-d762f20daea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:20.090417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0f3bd-520f-48b6-9e48-d762f20daea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:20.090417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0f3bd-520f-48b6-9e48-d762f20daea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:20.090417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0f3bd-520f-48b6-9e48-d762f20daea8 admin admin] PATH_INFO: `/v3/projects/a51ddeb469a64a0ca63268d50a01fe5c/users/3e591ce2e4354a89a064213d54ce10f6/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:20.116958 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0f3bd-520f-48b6-9e48-d762f20daea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a51ddeb469a64a0ca63268d50a01fe5c, user_id=3e591ce2e4354a89a064213d54ce10f6, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:20.119675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0f3bd-520f-48b6-9e48-d762f20daea8 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:20.208985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1263/2530] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:20 2022] PUT /identity/v3/projects/a51ddeb469a64a0ca63268d50a01fe5c/users/3e591ce2e4354a89a064213d54ce10f6/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:20.219236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78159117-3122-4340-be80-2be98bac130f None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:20.219529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78159117-3122-4340-be80-2be98bac130f None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:20.219813 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78159117-3122-4340-be80-2be98bac130f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:20.252983 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-78159117-3122-4340-be80-2be98bac130f None None] MFA Rules not processed for user `3e591ce2e4354a89a064213d54ce10f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:20.255047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-78159117-3122-4340-be80-2be98bac130f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:20.331499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1268/2531] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:20.343577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0f3bd-520f-48b6-9e48-d762f20daea8 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:20.345379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-19a0f3bd-520f-48b6-9e48-d762f20daea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:20.427810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0f3bd-520f-48b6-9e48-d762f20daea8 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:20.435647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03379e5e-ca76-4e40-9af9-8078174ca0b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:20.436285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03379e5e-ca76-4e40-9af9-8078174ca0b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:20.436599 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03379e5e-ca76-4e40-9af9-8078174ca0b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:20.436870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03379e5e-ca76-4e40-9af9-8078174ca0b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:20.446630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03379e5e-ca76-4e40-9af9-8078174ca0b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:20.450522 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03379e5e-ca76-4e40-9af9-8078174ca0b5 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:20.463906 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1264/2532] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:30.327766 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:30.456044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:33.161300 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78159117-3122-4340-be80-2be98bac130f None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:33.225321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78159117-3122-4340-be80-2be98bac130f None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:33.229864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:33.230496 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:33.230775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:33.231046 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] PATH_INFO: `/v3/users/3e591ce2e4354a89a064213d54ce10f6` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:33.232184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e591ce2e4354a89a064213d54ce10f6)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:33.234106 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:33.536987 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e591ce2e4354a89a064213d54ce10f6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:33.543754 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e591ce2e4354a89a064213d54ce10f6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:33.803229 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e591ce2e4354a89a064213d54ce10f6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:33.803806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e591ce2e4354a89a064213d54ce10f6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:33.817279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e591ce2e4354a89a064213d54ce10f6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:33.823103 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1269/2533] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:33 2022] DELETE /identity/v3/users/3e591ce2e4354a89a064213d54ce10f6 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:33.827584 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63deea4c-da75-4db8-9c71-a18625a39fc7 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:33.828087 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63deea4c-da75-4db8-9c71-a18625a39fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:33.828226 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63deea4c-da75-4db8-9c71-a18625a39fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:33.847572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-63deea4c-da75-4db8-9c71-a18625a39fc7 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:33.848797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-63deea4c-da75-4db8-9c71-a18625a39fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:33.925219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1265/2534] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:33.935154 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:33.936611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:33.994876 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ef60a-73ac-46f8-8cc6-c1a275663071 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:33.999867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e85e6-c2c7-44be-a4dc-4a302d541ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:34.000494 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e85e6-c2c7-44be-a4dc-4a302d541ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:34.000774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e85e6-c2c7-44be-a4dc-4a302d541ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:34.001043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e85e6-c2c7-44be-a4dc-4a302d541ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:34.016363 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e85e6-c2c7-44be-a4dc-4a302d541ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:34.017866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e85e6-c2c7-44be-a4dc-4a302d541ec6 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:34.028782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1270/2535] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:34.474755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63deea4c-da75-4db8-9c71-a18625a39fc7 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:34.476356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-63deea4c-da75-4db8-9c71-a18625a39fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:34.488853 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63deea4c-da75-4db8-9c71-a18625a39fc7 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:34.493782 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da97f69-4c83-4829-96f3-28de56c99eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:34.494468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da97f69-4c83-4829-96f3-28de56c99eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:34.494753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da97f69-4c83-4829-96f3-28de56c99eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:34.495026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da97f69-4c83-4829-96f3-28de56c99eb3 admin admin] PATH_INFO: `/v3/projects/a51ddeb469a64a0ca63268d50a01fe5c` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:34.496153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da97f69-4c83-4829-96f3-28de56c99eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a51ddeb469a64a0ca63268d50a01fe5c)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:34.498106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da97f69-4c83-4829-96f3-28de56c99eb3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:34.741336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5da97f69-4c83-4829-96f3-28de56c99eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a51ddeb469a64a0ca63268d50a01fe5c', 'request_id': 'req-5da97f69-4c83-4829-96f3-28de56c99eb3'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.746999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5da97f69-4c83-4829-96f3-28de56c99eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a51ddeb469a64a0ca63268d50a01fe5c', 'request_id': 'req-5da97f69-4c83-4829-96f3-28de56c99eb3'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.883560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5da97f69-4c83-4829-96f3-28de56c99eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a51ddeb469a64a0ca63268d50a01fe5c', 'request_id': 'req-5da97f69-4c83-4829-96f3-28de56c99eb3'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.895856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5da97f69-4c83-4829-96f3-28de56c99eb3 admin admin] The token cache is being invalidate because project a51ddeb469a64a0ca63268d50a01fe5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:34.896149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5da97f69-4c83-4829-96f3-28de56c99eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.896802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-5da97f69-4c83-4829-96f3-28de56c99eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.898566 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1266/2536] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:34 2022] DELETE /identity/v3/projects/a51ddeb469a64a0ca63268d50a01fe5c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:34.903409 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff8890-73b2-48a4-b204-5903faed2f22 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:34.903873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff8890-73b2-48a4-b204-5903faed2f22 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:34.903976 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff8890-73b2-48a4-b204-5903faed2f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:34.924058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-b8ff8890-73b2-48a4-b204-5903faed2f22 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:34.925841 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b8ff8890-73b2-48a4-b204-5903faed2f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:34.942980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:34.996963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1271/2537] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:35.004612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86812f8c-cb1c-4cef-acb0-09f8d11c66b8 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:35.004903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86812f8c-cb1c-4cef-acb0-09f8d11c66b8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:35.005186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86812f8c-cb1c-4cef-acb0-09f8d11c66b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:35.032532 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-86812f8c-cb1c-4cef-acb0-09f8d11c66b8 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:35.033917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-86812f8c-cb1c-4cef-acb0-09f8d11c66b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:35.043485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1267/2538] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:35.047070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff8890-73b2-48a4-b204-5903faed2f22 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:35.058728 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff8890-73b2-48a4-b204-5903faed2f22 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:35.065041 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b57be-511e-4587-9962-d2d8c327c015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:35.066315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b57be-511e-4587-9962-d2d8c327c015 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:35.066747 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b57be-511e-4587-9962-d2d8c327c015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:35.067789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b57be-511e-4587-9962-d2d8c327c015 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:35.068742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b57be-511e-4587-9962-d2d8c327c015 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:35.075301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b57be-511e-4587-9962-d2d8c327c015 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:35.075301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b57be-511e-4587-9962-d2d8c327c015 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:35.087063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1272/2539] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:35.087956 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86812f8c-cb1c-4cef-acb0-09f8d11c66b8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:35.088235 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b57be-511e-4587-9962-d2d8c327c015 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:35.089401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-86812f8c-cb1c-4cef-acb0-09f8d11c66b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:35.099811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b57be-511e-4587-9962-d2d8c327c015 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:35.104715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e39de-bf4a-4fc9-87d0-8ec483c3464f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:35.105321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e39de-bf4a-4fc9-87d0-8ec483c3464f admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:35.105603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e39de-bf4a-4fc9-87d0-8ec483c3464f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:35.105888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e39de-bf4a-4fc9-87d0-8ec483c3464f admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:35.106518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e39de-bf4a-4fc9-87d0-8ec483c3464f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:35.108462 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e39de-bf4a-4fc9-87d0-8ec483c3464f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:35.149868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86812f8c-cb1c-4cef-acb0-09f8d11c66b8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:35.153761 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f190332-06a9-4003-b5de-ba952817c636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:35.155207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f190332-06a9-4003-b5de-ba952817c636 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:35.155946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f190332-06a9-4003-b5de-ba952817c636 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:35.156317 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f190332-06a9-4003-b5de-ba952817c636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:35.158155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3f190332-06a9-4003-b5de-ba952817c636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:35.168855 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f190332-06a9-4003-b5de-ba952817c636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:35.170328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f190332-06a9-4003-b5de-ba952817c636 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:35.179898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1268/2540] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:35.193807 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1273/2541] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:35 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:35.198481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f190332-06a9-4003-b5de-ba952817c636 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:35.210274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f190332-06a9-4003-b5de-ba952817c636 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:35.227208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:35.266436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6e7e-dd29-45e8-ae2d-d5c07258cdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:35.267047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6e7e-dd29-45e8-ae2d-d5c07258cdb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:35.267325 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6e7e-dd29-45e8-ae2d-d5c07258cdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:35.267594 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6e7e-dd29-45e8-ae2d-d5c07258cdb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:35.268892 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b6e7e-dd29-45e8-ae2d-d5c07258cdb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:35.271804 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b6e7e-dd29-45e8-ae2d-d5c07258cdb6 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:35.449386 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1269/2542] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:35 2022] POST /identity/v3/users => generated 283 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:35.453770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e39de-bf4a-4fc9-87d0-8ec483c3464f admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:35.464527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e39de-bf4a-4fc9-87d0-8ec483c3464f admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:35.469307 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d6442-44ec-4ce4-b270-c908e86b8e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:35.469933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d6442-44ec-4ce4-b270-c908e86b8e5d admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:35.470253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d6442-44ec-4ce4-b270-c908e86b8e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:35.470533 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d6442-44ec-4ce4-b270-c908e86b8e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:35.471180 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d6442-44ec-4ce4-b270-c908e86b8e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:35.472808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d6442-44ec-4ce4-b270-c908e86b8e5d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:35.484020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1274/2543] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:35.488975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6e7e-dd29-45e8-ae2d-d5c07258cdb6 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:35.499161 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6e7e-dd29-45e8-ae2d-d5c07258cdb6 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:35.504615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f75ab1-4393-48d6-a8cb-1619e78c6e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:35.505331 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f75ab1-4393-48d6-a8cb-1619e78c6e02 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:35.505683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f75ab1-4393-48d6-a8cb-1619e78c6e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:35.505971 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f75ab1-4393-48d6-a8cb-1619e78c6e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:35.506615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f75ab1-4393-48d6-a8cb-1619e78c6e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:35.508227 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f75ab1-4393-48d6-a8cb-1619e78c6e02 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:35.518839 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1270/2544] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:35.523315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d6442-44ec-4ce4-b270-c908e86b8e5d admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:35.533902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d6442-44ec-4ce4-b270-c908e86b8e5d admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:35.538534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190714fc-2f25-42ab-a22d-fc93371fc756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:35.539220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190714fc-2f25-42ab-a22d-fc93371fc756 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:35.539504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190714fc-2f25-42ab-a22d-fc93371fc756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:35.539781 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190714fc-2f25-42ab-a22d-fc93371fc756 admin admin] PATH_INFO: `/v3/projects/d90e46bdabd5478c9db2e2b32539b1ee/users/8437ce32d6ee44ebaeb0eccd08c4dca8/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:35.554931 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190714fc-2f25-42ab-a22d-fc93371fc756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d90e46bdabd5478c9db2e2b32539b1ee, user_id=8437ce32d6ee44ebaeb0eccd08c4dca8, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:35.557623 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190714fc-2f25-42ab-a22d-fc93371fc756 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:35.701011 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1275/2545] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:35 2022] PUT /identity/v3/projects/d90e46bdabd5478c9db2e2b32539b1ee/users/8437ce32d6ee44ebaeb0eccd08c4dca8/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:35.712701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36118b78-9c1c-4c1a-8b36-1a0378f763b1 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:35.712999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36118b78-9c1c-4c1a-8b36-1a0378f763b1 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:35.713267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36118b78-9c1c-4c1a-8b36-1a0378f763b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:35.744159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-36118b78-9c1c-4c1a-8b36-1a0378f763b1 None None] MFA Rules not processed for user `8437ce32d6ee44ebaeb0eccd08c4dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:35.745552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-36118b78-9c1c-4c1a-8b36-1a0378f763b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:35.816864 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1271/2546] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:35.821711 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190714fc-2f25-42ab-a22d-fc93371fc756 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:35.891192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190714fc-2f25-42ab-a22d-fc93371fc756 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:35.896450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ae08b-247f-4177-8d82-46af4906ab57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:35.897048 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ae08b-247f-4177-8d82-46af4906ab57 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:35.897585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ae08b-247f-4177-8d82-46af4906ab57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:35.897866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ae08b-247f-4177-8d82-46af4906ab57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:35.898492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ae08b-247f-4177-8d82-46af4906ab57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:35.900401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ae08b-247f-4177-8d82-46af4906ab57 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.156879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1276/2547] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:35 2022] POST /identity/v3/projects => generated 365 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:36.158032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36118b78-9c1c-4c1a-8b36-1a0378f763b1 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.168993 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36118b78-9c1c-4c1a-8b36-1a0378f763b1 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.225086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6171a-8f3e-4732-a8f9-a68dad6cc89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.225693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6171a-8f3e-4732-a8f9-a68dad6cc89e admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.225977 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6171a-8f3e-4732-a8f9-a68dad6cc89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.226271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6171a-8f3e-4732-a8f9-a68dad6cc89e admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.226888 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6171a-8f3e-4732-a8f9-a68dad6cc89e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.228850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6171a-8f3e-4732-a8f9-a68dad6cc89e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.368630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1272/2548] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:36 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:36.372451 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ae08b-247f-4177-8d82-46af4906ab57 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.383012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ae08b-247f-4177-8d82-46af4906ab57 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.387511 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b839ea-9f68-4fc5-a681-5e31ea203d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.388100 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b839ea-9f68-4fc5-a681-5e31ea203d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.388409 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b839ea-9f68-4fc5-a681-5e31ea203d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.388701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b839ea-9f68-4fc5-a681-5e31ea203d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.389327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b839ea-9f68-4fc5-a681-5e31ea203d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.391003 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b839ea-9f68-4fc5-a681-5e31ea203d07 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.402120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1277/2549] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:36.414457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6171a-8f3e-4732-a8f9-a68dad6cc89e admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.436694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6171a-8f3e-4732-a8f9-a68dad6cc89e admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.443308 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcebff-6c3b-456b-a12e-237f7fb814c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.444110 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcebff-6c3b-456b-a12e-237f7fb814c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.444504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcebff-6c3b-456b-a12e-237f7fb814c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.445033 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcebff-6c3b-456b-a12e-237f7fb814c3 admin admin] PATH_INFO: `/v3/projects/a4cba21a480e46f8bfb6077c70b50a2b/users/696f445b6342450fb7432baa0387628e/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.462923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcebff-6c3b-456b-a12e-237f7fb814c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4cba21a480e46f8bfb6077c70b50a2b, user_id=696f445b6342450fb7432baa0387628e, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.465741 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcebff-6c3b-456b-a12e-237f7fb814c3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.468207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:36.583056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1273/2550] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:36 2022] PUT /identity/v3/projects/a4cba21a480e46f8bfb6077c70b50a2b/users/696f445b6342450fb7432baa0387628e/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:36.586521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b839ea-9f68-4fc5-a681-5e31ea203d07 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.599834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b839ea-9f68-4fc5-a681-5e31ea203d07 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.635795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:36.669235 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde76134-be7a-48fb-aa44-ceaa3ca509f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.675066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde76134-be7a-48fb-aa44-ceaa3ca509f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.675066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde76134-be7a-48fb-aa44-ceaa3ca509f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.675066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde76134-be7a-48fb-aa44-ceaa3ca509f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.675066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde76134-be7a-48fb-aa44-ceaa3ca509f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.675066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde76134-be7a-48fb-aa44-ceaa3ca509f0 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.684290 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1278/2551] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:36.688381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcebff-6c3b-456b-a12e-237f7fb814c3 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.745303 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcebff-6c3b-456b-a12e-237f7fb814c3 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.745303 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59598d6-7e45-47c9-b616-44ac705950c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.745303 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59598d6-7e45-47c9-b616-44ac705950c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.745303 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59598d6-7e45-47c9-b616-44ac705950c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.745696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59598d6-7e45-47c9-b616-44ac705950c5 admin admin] PATH_INFO: `/v3/projects/a4cba21a480e46f8bfb6077c70b50a2b/users/696f445b6342450fb7432baa0387628e/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.745696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59598d6-7e45-47c9-b616-44ac705950c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4cba21a480e46f8bfb6077c70b50a2b, user_id=696f445b6342450fb7432baa0387628e, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.745696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59598d6-7e45-47c9-b616-44ac705950c5 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.745696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1274/2552] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:36 2022] PUT /identity/v3/projects/a4cba21a480e46f8bfb6077c70b50a2b/users/696f445b6342450fb7432baa0387628e/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:36.746076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bb5ee-0e63-4b6e-9a59-68fa13667a73 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.746076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bb5ee-0e63-4b6e-9a59-68fa13667a73 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.746076 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bb5ee-0e63-4b6e-9a59-68fa13667a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.764732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-774bb5ee-0e63-4b6e-9a59-68fa13667a73 None None] MFA Rules not processed for user `696f445b6342450fb7432baa0387628e`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:36.765943 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-774bb5ee-0e63-4b6e-9a59-68fa13667a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:36.838105 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1279/2553] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:36.849302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59598d6-7e45-47c9-b616-44ac705950c5 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.850996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c59598d6-7e45-47c9-b616-44ac705950c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:36.909640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59598d6-7e45-47c9-b616-44ac705950c5 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.913156 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f44d7-d8a6-47fb-b8b3-036b10a7229b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.913831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f44d7-d8a6-47fb-b8b3-036b10a7229b service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.914223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f44d7-d8a6-47fb-b8b3-036b10a7229b service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.914504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f44d7-d8a6-47fb-b8b3-036b10a7229b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.925385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f44d7-d8a6-47fb-b8b3-036b10a7229b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.926339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f44d7-d8a6-47fb-b8b3-036b10a7229b service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.937176 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1275/2554] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:37.204435 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bb5ee-0e63-4b6e-9a59-68fa13667a73 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:37.214527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bb5ee-0e63-4b6e-9a59-68fa13667a73 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:37.218303 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5605f0e-63c0-4fff-80ab-dbe739e243fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:37.218927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5605f0e-63c0-4fff-80ab-dbe739e243fe service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:37.219216 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5605f0e-63c0-4fff-80ab-dbe739e243fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:37.219495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5605f0e-63c0-4fff-80ab-dbe739e243fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:37.281190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5605f0e-63c0-4fff-80ab-dbe739e243fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:37.282687 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5605f0e-63c0-4fff-80ab-dbe739e243fe service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:37.293710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1280/2555] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:39.536627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf7600-34b7-441a-8c7f-9946a02fe65e None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:39.537120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf7600-34b7-441a-8c7f-9946a02fe65e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:39.537221 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf7600-34b7-441a-8c7f-9946a02fe65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:39.555875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-8fcf7600-34b7-441a-8c7f-9946a02fe65e None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:39.557129 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8fcf7600-34b7-441a-8c7f-9946a02fe65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:39.628379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1276/2556] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:34:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:39.641047 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5605f0e-63c0-4fff-80ab-dbe739e243fe service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:39.642490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d5605f0e-63c0-4fff-80ab-dbe739e243fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:39.692577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:39.713438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5605f0e-63c0-4fff-80ab-dbe739e243fe service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:39.718702 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3b281-3f7e-4865-8e7d-e6f40f14782b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:39.719301 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3b281-3f7e-4865-8e7d-e6f40f14782b service swift] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:39.719581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3b281-3f7e-4865-8e7d-e6f40f14782b service swift] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:39.719856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3b281-3f7e-4865-8e7d-e6f40f14782b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:39.729254 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3b281-3f7e-4865-8e7d-e6f40f14782b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:39.738031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3b281-3f7e-4865-8e7d-e6f40f14782b service swift] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:39.740901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1281/2557] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:34:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 20:34:41.625830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf7600-34b7-441a-8c7f-9946a02fe65e None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:41.739723 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf7600-34b7-441a-8c7f-9946a02fe65e None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:41.746117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:41.746507 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:41.746790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:41.747067 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] PATH_INFO: `/v3/users/8437ce32d6ee44ebaeb0eccd08c4dca8` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:41.748401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8437ce32d6ee44ebaeb0eccd08c4dca8)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:41.751488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:41.930078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:41.945920 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8437ce32d6ee44ebaeb0eccd08c4dca8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:41.946910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8437ce32d6ee44ebaeb0eccd08c4dca8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:41.954262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8437ce32d6ee44ebaeb0eccd08c4dca8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:42.066446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8437ce32d6ee44ebaeb0eccd08c4dca8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:42.067199 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8437ce32d6ee44ebaeb0eccd08c4dca8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:42.093269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1277/2558] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:41 2022] DELETE /identity/v3/users/8437ce32d6ee44ebaeb0eccd08c4dca8 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:42.108143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236bc91-d445-4440-954e-10f6153740cd None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:42.108143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236bc91-d445-4440-954e-10f6153740cd None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:42.108143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236bc91-d445-4440-954e-10f6153740cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:42.119471 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-6236bc91-d445-4440-954e-10f6153740cd None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:42.121080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-6236bc91-d445-4440-954e-10f6153740cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:42.203212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1282/2559] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:42.213310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:42.214746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:42.279627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16477c02-dce7-47c5-8f22-9a0b2989e900 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:42.283764 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522b4b6-85f2-4d7f-a387-962a7894aebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:42.284627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b4b6-85f2-4d7f-a387-962a7894aebc service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:42.285143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b4b6-85f2-4d7f-a387-962a7894aebc service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:42.285615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b4b6-85f2-4d7f-a387-962a7894aebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:42.295753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522b4b6-85f2-4d7f-a387-962a7894aebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:42.297437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522b4b6-85f2-4d7f-a387-962a7894aebc service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:42.308149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1278/2560] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:42.870981 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236bc91-d445-4440-954e-10f6153740cd None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:42.873009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-6236bc91-d445-4440-954e-10f6153740cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:42.899536 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236bc91-d445-4440-954e-10f6153740cd None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:42.912875 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:42.913615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:42.913781 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:42.914061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] PATH_INFO: `/v3/projects/d90e46bdabd5478c9db2e2b32539b1ee` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:42.915220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d90e46bdabd5478c9db2e2b32539b1ee)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:42.917219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:43.183334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd90e46bdabd5478c9db2e2b32539b1ee', 'request_id': 'req-ded333c7-27dc-4289-91d9-c5cd9d7651b4'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:43.191830 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd90e46bdabd5478c9db2e2b32539b1ee', 'request_id': 'req-ded333c7-27dc-4289-91d9-c5cd9d7651b4'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:43.198527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd90e46bdabd5478c9db2e2b32539b1ee', 'request_id': 'req-ded333c7-27dc-4289-91d9-c5cd9d7651b4'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:43.387838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] The token cache is being invalidate because project d90e46bdabd5478c9db2e2b32539b1ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:43.388295 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:43.388765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:43.390274 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1283/2561] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:42 2022] DELETE /identity/v3/projects/d90e46bdabd5478c9db2e2b32539b1ee => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:43.395643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522b4b6-85f2-4d7f-a387-962a7894aebc service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:43.407149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522b4b6-85f2-4d7f-a387-962a7894aebc service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:43.468186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ebf8c-7a07-491c-b9dc-cb20773e84b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:43.468837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ebf8c-7a07-491c-b9dc-cb20773e84b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:43.469122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ebf8c-7a07-491c-b9dc-cb20773e84b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:43.469464 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ebf8c-7a07-491c-b9dc-cb20773e84b5 admin admin] PATH_INFO: `/v3/users/696f445b6342450fb7432baa0387628e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:43.470627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ebf8c-7a07-491c-b9dc-cb20773e84b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=696f445b6342450fb7432baa0387628e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:43.472634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ebf8c-7a07-491c-b9dc-cb20773e84b5 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:43.733739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b5ebf8c-7a07-491c-b9dc-cb20773e84b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '696f445b6342450fb7432baa0387628e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:43.734488 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b5ebf8c-7a07-491c-b9dc-cb20773e84b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '696f445b6342450fb7432baa0387628e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:43.739868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:43.742809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b5ebf8c-7a07-491c-b9dc-cb20773e84b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '696f445b6342450fb7432baa0387628e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:43.837883 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b5ebf8c-7a07-491c-b9dc-cb20773e84b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '696f445b6342450fb7432baa0387628e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:43.838533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b5ebf8c-7a07-491c-b9dc-cb20773e84b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '696f445b6342450fb7432baa0387628e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:43.858064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1279/2562] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:43 2022] DELETE /identity/v3/users/696f445b6342450fb7432baa0387628e => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:44.417946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:44.419754 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:44.435626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded333c7-27dc-4289-91d9-c5cd9d7651b4 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:44.509540 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09ca3b-79dc-4b85-9187-b10943f61731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:44.510098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09ca3b-79dc-4b85-9187-b10943f61731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:44.510380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09ca3b-79dc-4b85-9187-b10943f61731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:44.510706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09ca3b-79dc-4b85-9187-b10943f61731 admin admin] PATH_INFO: `/v3/projects/a4cba21a480e46f8bfb6077c70b50a2b` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:44.512070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09ca3b-79dc-4b85-9187-b10943f61731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4cba21a480e46f8bfb6077c70b50a2b)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:44.514190 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09ca3b-79dc-4b85-9187-b10943f61731 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:44.516139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:44.663659 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fb09ca3b-79dc-4b85-9187-b10943f61731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4cba21a480e46f8bfb6077c70b50a2b', 'request_id': 'req-fb09ca3b-79dc-4b85-9187-b10943f61731'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:44.670292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fb09ca3b-79dc-4b85-9187-b10943f61731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4cba21a480e46f8bfb6077c70b50a2b', 'request_id': 'req-fb09ca3b-79dc-4b85-9187-b10943f61731'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:44.676522 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fb09ca3b-79dc-4b85-9187-b10943f61731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4cba21a480e46f8bfb6077c70b50a2b', 'request_id': 'req-fb09ca3b-79dc-4b85-9187-b10943f61731'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:44.981580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fb09ca3b-79dc-4b85-9187-b10943f61731 admin admin] The token cache is being invalidate because project a4cba21a480e46f8bfb6077c70b50a2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:44.998359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fb09ca3b-79dc-4b85-9187-b10943f61731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:44.998359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fb09ca3b-79dc-4b85-9187-b10943f61731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:44.998359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1284/2563] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:44 2022] DELETE /identity/v3/projects/a4cba21a480e46f8bfb6077c70b50a2b => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:45.001970 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f3fe0-dc48-49c9-a74d-0bc872397500 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:45.002528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f3fe0-dc48-49c9-a74d-0bc872397500 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:45.002528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f3fe0-dc48-49c9-a74d-0bc872397500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:45.023397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-246f3fe0-dc48-49c9-a74d-0bc872397500 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:45.024635 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-246f3fe0-dc48-49c9-a74d-0bc872397500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:45.104263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1280/2564] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:45.105980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec9723-9074-4340-b271-572801774128 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:45.106252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec9723-9074-4340-b271-572801774128 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:45.106534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec9723-9074-4340-b271-572801774128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:45.146131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-67ec9723-9074-4340-b271-572801774128 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:45.147639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-67ec9723-9074-4340-b271-572801774128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:45.169310 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1285/2565] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:45.170908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f3fe0-dc48-49c9-a74d-0bc872397500 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:45.185822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f3fe0-dc48-49c9-a74d-0bc872397500 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:45.192023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e635b9-a08b-4207-b17d-8c9bf523f1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:45.193066 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e635b9-a08b-4207-b17d-8c9bf523f1ad admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:45.193612 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e635b9-a08b-4207-b17d-8c9bf523f1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:45.194118 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e635b9-a08b-4207-b17d-8c9bf523f1ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:45.195363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e635b9-a08b-4207-b17d-8c9bf523f1ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:45.195752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e635b9-a08b-4207-b17d-8c9bf523f1ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:45.197510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e635b9-a08b-4207-b17d-8c9bf523f1ad admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:45.212203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1281/2566] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:45.215905 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec9723-9074-4340-b271-572801774128 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:45.225701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec9723-9074-4340-b271-572801774128 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:45.230748 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299eb479-6efd-42bb-bb03-07f3f1c29548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:45.231358 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299eb479-6efd-42bb-bb03-07f3f1c29548 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:45.231637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299eb479-6efd-42bb-bb03-07f3f1c29548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:45.231909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299eb479-6efd-42bb-bb03-07f3f1c29548 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:45.232729 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299eb479-6efd-42bb-bb03-07f3f1c29548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:45.234712 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299eb479-6efd-42bb-bb03-07f3f1c29548 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:45.350824 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1286/2567] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:45 2022] POST /identity/v3/projects => generated 391 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:45.355881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e635b9-a08b-4207-b17d-8c9bf523f1ad admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:45.367581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e635b9-a08b-4207-b17d-8c9bf523f1ad admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:45.468767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90d7d1-39ff-4964-8cc9-cfa2c202f81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:45.470802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90d7d1-39ff-4964-8cc9-cfa2c202f81e admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:45.470802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90d7d1-39ff-4964-8cc9-cfa2c202f81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:45.470802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90d7d1-39ff-4964-8cc9-cfa2c202f81e admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:45.470802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90d7d1-39ff-4964-8cc9-cfa2c202f81e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:45.473976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90d7d1-39ff-4964-8cc9-cfa2c202f81e admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:45.628685 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1282/2568] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:45 2022] POST /identity/v3/users => generated 296 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:45.631327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299eb479-6efd-42bb-bb03-07f3f1c29548 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:45.641764 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299eb479-6efd-42bb-bb03-07f3f1c29548 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:45.646806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479672a-ccb0-46a1-896a-5917b47f31a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:45.647777 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479672a-ccb0-46a1-896a-5917b47f31a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:45.648062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479672a-ccb0-46a1-896a-5917b47f31a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:45.648371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479672a-ccb0-46a1-896a-5917b47f31a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:45.649029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479672a-ccb0-46a1-896a-5917b47f31a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:45.650634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479672a-ccb0-46a1-896a-5917b47f31a2 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:45.665072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1287/2569] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:45.675363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90d7d1-39ff-4964-8cc9-cfa2c202f81e admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:45.686418 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90d7d1-39ff-4964-8cc9-cfa2c202f81e admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:45.692039 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed0068-0295-46e6-8d70-5487aa8d94d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:45.692688 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed0068-0295-46e6-8d70-5487aa8d94d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:45.692991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed0068-0295-46e6-8d70-5487aa8d94d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:45.693282 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed0068-0295-46e6-8d70-5487aa8d94d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:45.693950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed0068-0295-46e6-8d70-5487aa8d94d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:45.695618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed0068-0295-46e6-8d70-5487aa8d94d5 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:45.713501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1283/2570] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:45.717450 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479672a-ccb0-46a1-896a-5917b47f31a2 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:45.725900 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479672a-ccb0-46a1-896a-5917b47f31a2 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:45.731505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39192b97-9240-4467-bc13-397db2d7e749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:45.732158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39192b97-9240-4467-bc13-397db2d7e749 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:45.732472 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39192b97-9240-4467-bc13-397db2d7e749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:45.732852 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39192b97-9240-4467-bc13-397db2d7e749 admin admin] PATH_INFO: `/v3/projects/abd4577b5ab4419d8e4d714e69ded4fd/users/33f628108d834602afcddaa9bc283b7f/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:45.749632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39192b97-9240-4467-bc13-397db2d7e749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd4577b5ab4419d8e4d714e69ded4fd, user_id=33f628108d834602afcddaa9bc283b7f, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:45.752874 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39192b97-9240-4467-bc13-397db2d7e749 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:45.866817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1288/2571] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:45 2022] PUT /identity/v3/projects/abd4577b5ab4419d8e4d714e69ded4fd/users/33f628108d834602afcddaa9bc283b7f/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:45.880635 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543f62b-3077-4456-ad2a-04f6087d6a3e None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:45.881078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543f62b-3077-4456-ad2a-04f6087d6a3e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:45.881358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543f62b-3077-4456-ad2a-04f6087d6a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:45.927454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-a543f62b-3077-4456-ad2a-04f6087d6a3e None None] MFA Rules not processed for user `33f628108d834602afcddaa9bc283b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:45.928998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a543f62b-3077-4456-ad2a-04f6087d6a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:45.957806 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:45.999643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1284/2572] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:46.004576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39192b97-9240-4467-bc13-397db2d7e749 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:46.074915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39192b97-9240-4467-bc13-397db2d7e749 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:46.080853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50e1dd-e383-4fb8-8845-dcf7e4f05764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:46.081552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50e1dd-e383-4fb8-8845-dcf7e4f05764 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:46.081927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50e1dd-e383-4fb8-8845-dcf7e4f05764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:46.082271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50e1dd-e383-4fb8-8845-dcf7e4f05764 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:46.082955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50e1dd-e383-4fb8-8845-dcf7e4f05764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:46.085130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50e1dd-e383-4fb8-8845-dcf7e4f05764 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:46.234787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1289/2573] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:46 2022] POST /identity/v3/projects => generated 391 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:46.239792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543f62b-3077-4456-ad2a-04f6087d6a3e None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:46.252036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543f62b-3077-4456-ad2a-04f6087d6a3e None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:46.346837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085c60f-5f99-45dc-acb7-906af3dcd21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:46.347655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085c60f-5f99-45dc-acb7-906af3dcd21a admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:46.348031 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085c60f-5f99-45dc-acb7-906af3dcd21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:46.348431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085c60f-5f99-45dc-acb7-906af3dcd21a admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:46.349174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085c60f-5f99-45dc-acb7-906af3dcd21a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:46.351876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085c60f-5f99-45dc-acb7-906af3dcd21a admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:46.475264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1285/2574] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:46 2022] POST /identity/v3/users => generated 302 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:46.480081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50e1dd-e383-4fb8-8845-dcf7e4f05764 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:46.491480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50e1dd-e383-4fb8-8845-dcf7e4f05764 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:46.496468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351cf18-11e6-4750-9632-04ccfe965b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:46.497116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351cf18-11e6-4750-9632-04ccfe965b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:46.497415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351cf18-11e6-4750-9632-04ccfe965b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:46.497689 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351cf18-11e6-4750-9632-04ccfe965b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:46.498377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351cf18-11e6-4750-9632-04ccfe965b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:46.500106 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351cf18-11e6-4750-9632-04ccfe965b84 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:46.507337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:46.518637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1290/2575] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:46.520400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085c60f-5f99-45dc-acb7-906af3dcd21a admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:46.534113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085c60f-5f99-45dc-acb7-906af3dcd21a admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:46.539367 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed676389-0fee-4615-8561-8c72870b1487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:46.540668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed676389-0fee-4615-8561-8c72870b1487 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:46.541053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed676389-0fee-4615-8561-8c72870b1487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:46.541228 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed676389-0fee-4615-8561-8c72870b1487 admin admin] PATH_INFO: `/v3/projects/3b3072ccd6144c7583c306ba00cc86a0/users/771cbd58af4f40afb427b7deeb9a9d3c/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:46.567314 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed676389-0fee-4615-8561-8c72870b1487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3072ccd6144c7583c306ba00cc86a0, user_id=771cbd58af4f40afb427b7deeb9a9d3c, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:46.570695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed676389-0fee-4615-8561-8c72870b1487 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:46.696921 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1286/2576] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:46 2022] PUT /identity/v3/projects/3b3072ccd6144c7583c306ba00cc86a0/users/771cbd58af4f40afb427b7deeb9a9d3c/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:46.701806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351cf18-11e6-4750-9632-04ccfe965b84 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:46.712743 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351cf18-11e6-4750-9632-04ccfe965b84 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:46.784359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978e47b-3e05-4ca1-8f58-99b309b183b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:46.785972 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978e47b-3e05-4ca1-8f58-99b309b183b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:46.786479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978e47b-3e05-4ca1-8f58-99b309b183b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:46.786980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978e47b-3e05-4ca1-8f58-99b309b183b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:46.787876 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978e47b-3e05-4ca1-8f58-99b309b183b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:46.791106 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978e47b-3e05-4ca1-8f58-99b309b183b7 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:46.805020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1291/2577] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:46.809237 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed676389-0fee-4615-8561-8c72870b1487 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:46.838290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed676389-0fee-4615-8561-8c72870b1487 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:46.843403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e820eac-c291-4595-b064-92b9d4e39b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:46.844103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e820eac-c291-4595-b064-92b9d4e39b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:46.844639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e820eac-c291-4595-b064-92b9d4e39b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:46.844923 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e820eac-c291-4595-b064-92b9d4e39b57 admin admin] PATH_INFO: `/v3/projects/3b3072ccd6144c7583c306ba00cc86a0/users/771cbd58af4f40afb427b7deeb9a9d3c/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:46.847212 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e820eac-c291-4595-b064-92b9d4e39b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3072ccd6144c7583c306ba00cc86a0, user_id=771cbd58af4f40afb427b7deeb9a9d3c, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:46.850046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e820eac-c291-4595-b064-92b9d4e39b57 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:46.864029 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1287/2578] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:46 2022] PUT /identity/v3/projects/3b3072ccd6144c7583c306ba00cc86a0/users/771cbd58af4f40afb427b7deeb9a9d3c/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:46.873283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35374d8-b606-44df-bdf4-e16cbf47f9c4 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:46.873692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35374d8-b606-44df-bdf4-e16cbf47f9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:46.874044 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35374d8-b606-44df-bdf4-e16cbf47f9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:46.905985 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e35374d8-b606-44df-bdf4-e16cbf47f9c4 None None] MFA Rules not processed for user `771cbd58af4f40afb427b7deeb9a9d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:46.907385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e35374d8-b606-44df-bdf4-e16cbf47f9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:47.003587 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1292/2579] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:46 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:47.015931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e820eac-c291-4595-b064-92b9d4e39b57 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:47.018963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5e820eac-c291-4595-b064-92b9d4e39b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:47.095206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e820eac-c291-4595-b064-92b9d4e39b57 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:47.099966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6460f9e-de3a-40ff-9bbf-6efb187ac493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:47.103208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6460f9e-de3a-40ff-9bbf-6efb187ac493 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:47.103208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6460f9e-de3a-40ff-9bbf-6efb187ac493 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:47.103208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6460f9e-de3a-40ff-9bbf-6efb187ac493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:47.112195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6460f9e-de3a-40ff-9bbf-6efb187ac493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:47.113811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6460f9e-de3a-40ff-9bbf-6efb187ac493 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:47.126289 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1288/2580] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:47 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:47.176851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35374d8-b606-44df-bdf4-e16cbf47f9c4 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:47.178904 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e35374d8-b606-44df-bdf4-e16cbf47f9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:47.192303 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35374d8-b606-44df-bdf4-e16cbf47f9c4 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:47.196560 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf4126-fcb1-487c-8e27-25897b57ca97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:47.197324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf4126-fcb1-487c-8e27-25897b57ca97 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:47.197890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf4126-fcb1-487c-8e27-25897b57ca97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:47.198246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf4126-fcb1-487c-8e27-25897b57ca97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:47.268141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf4126-fcb1-487c-8e27-25897b57ca97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:47.269644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf4126-fcb1-487c-8e27-25897b57ca97 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:47.279820 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1293/2581] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:54.175103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6460f9e-de3a-40ff-9bbf-6efb187ac493 service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:54.196837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:54.238757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6460f9e-de3a-40ff-9bbf-6efb187ac493 service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:54.243396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752122a-ed75-4776-9978-4402564b362d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:54.244029 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752122a-ed75-4776-9978-4402564b362d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:54.244347 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752122a-ed75-4776-9978-4402564b362d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:54.244630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752122a-ed75-4776-9978-4402564b362d admin admin] PATH_INFO: `/v3/users/33f628108d834602afcddaa9bc283b7f` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:54.245765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752122a-ed75-4776-9978-4402564b362d admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f628108d834602afcddaa9bc283b7f)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:54.247724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752122a-ed75-4776-9978-4402564b362d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:54.481946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13b0d7-a2e7-46c1-8a4b-f4a1040e5584 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:54.482476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13b0d7-a2e7-46c1-8a4b-f4a1040e5584 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:54.482476 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13b0d7-a2e7-46c1-8a4b-f4a1040e5584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:54.505449 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2752122a-ed75-4776-9978-4402564b362d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f628108d834602afcddaa9bc283b7f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:54.506102 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2752122a-ed75-4776-9978-4402564b362d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f628108d834602afcddaa9bc283b7f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:54.506236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-6f13b0d7-a2e7-46c1-8a4b-f4a1040e5584 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:54.507434 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-6f13b0d7-a2e7-46c1-8a4b-f4a1040e5584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:54.513704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2752122a-ed75-4776-9978-4402564b362d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f628108d834602afcddaa9bc283b7f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:54.521009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:54.565431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1294/2582] 10.222.0.85 () {58 vars in 967 bytes} [Mon Aug 29 20:34:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:54.577552 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13b0d7-a2e7-46c1-8a4b-f4a1040e5584 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:54.578875 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-6f13b0d7-a2e7-46c1-8a4b-f4a1040e5584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:54.648279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13b0d7-a2e7-46c1-8a4b-f4a1040e5584 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:54.654359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a863ad-6c91-475b-97c6-bae2672a6bdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:54.666884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a863ad-6c91-475b-97c6-bae2672a6bdc service swift] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:54.666884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a863ad-6c91-475b-97c6-bae2672a6bdc service swift] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:54.666884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a863ad-6c91-475b-97c6-bae2672a6bdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:54.666884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a863ad-6c91-475b-97c6-bae2672a6bdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:54.667978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a863ad-6c91-475b-97c6-bae2672a6bdc service swift] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:54.676528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1295/2583] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:34:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 20:34:54.677650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d9e50-6028-469c-91f3-a7d414eadcfe None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:54.678004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d9e50-6028-469c-91f3-a7d414eadcfe None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:54.678224 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d9e50-6028-469c-91f3-a7d414eadcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:54.697760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-576d9e50-6028-469c-91f3-a7d414eadcfe None None] MFA Rules not processed for user `75a2f770cc774d61baa6c07532665477`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:54.698695 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-576d9e50-6028-469c-91f3-a7d414eadcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:54.711002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2752122a-ed75-4776-9978-4402564b362d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f628108d834602afcddaa9bc283b7f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:54.711634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2752122a-ed75-4776-9978-4402564b362d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f628108d834602afcddaa9bc283b7f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:54.741354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1289/2584] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:54 2022] DELETE /identity/v3/users/33f628108d834602afcddaa9bc283b7f => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:54.742775 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac64d3c-23ab-4ac0-ba38-9ac46d068e83 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:54.743054 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac64d3c-23ab-4ac0-ba38-9ac46d068e83 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:54.743325 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac64d3c-23ab-4ac0-ba38-9ac46d068e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:54.764205 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1296/2585] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:54.764437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-fac64d3c-23ab-4ac0-ba38-9ac46d068e83 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:54.765122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-fac64d3c-23ab-4ac0-ba38-9ac46d068e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:54.826409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1290/2586] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:54.836967 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d9e50-6028-469c-91f3-a7d414eadcfe None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:54.840373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-576d9e50-6028-469c-91f3-a7d414eadcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:54.898113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d9e50-6028-469c-91f3-a7d414eadcfe None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:54.957115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7c136-0517-4936-86b7-20fa1b776a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:54.957986 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7c136-0517-4936-86b7-20fa1b776a1c service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:54.958174 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7c136-0517-4936-86b7-20fa1b776a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:54.958447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7c136-0517-4936-86b7-20fa1b776a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:54.965821 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:54.969454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7c136-0517-4936-86b7-20fa1b776a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:54.971014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7c136-0517-4936-86b7-20fa1b776a1c service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:54.990242 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1297/2587] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:55.583796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac64d3c-23ab-4ac0-ba38-9ac46d068e83 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:55.585558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-fac64d3c-23ab-4ac0-ba38-9ac46d068e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:55.600073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac64d3c-23ab-4ac0-ba38-9ac46d068e83 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:55.606010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:55.606764 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:55.607228 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:55.607356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] PATH_INFO: `/v3/projects/abd4577b5ab4419d8e4d714e69ded4fd` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:55.608723 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abd4577b5ab4419d8e4d714e69ded4fd)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:55.618672 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:55.823016 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:55.833170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abd4577b5ab4419d8e4d714e69ded4fd', 'request_id': 'req-903fb59a-f614-4b4e-bb82-d93c217b0223'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:55.838922 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abd4577b5ab4419d8e4d714e69ded4fd', 'request_id': 'req-903fb59a-f614-4b4e-bb82-d93c217b0223'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:55.923117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abd4577b5ab4419d8e4d714e69ded4fd', 'request_id': 'req-903fb59a-f614-4b4e-bb82-d93c217b0223'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:55.935211 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] The token cache is being invalidate because project abd4577b5ab4419d8e4d714e69ded4fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:55.935530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:55.936199 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:55.937831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1291/2588] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:55 2022] DELETE /identity/v3/projects/abd4577b5ab4419d8e4d714e69ded4fd => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:55.941845 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7c136-0517-4936-86b7-20fa1b776a1c service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:55.952339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7c136-0517-4936-86b7-20fa1b776a1c service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:56.010829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff9771-65e9-443f-8ea6-97ebd3ded0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:56.011467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff9771-65e9-443f-8ea6-97ebd3ded0c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:56.011757 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff9771-65e9-443f-8ea6-97ebd3ded0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:56.012028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff9771-65e9-443f-8ea6-97ebd3ded0c8 admin admin] PATH_INFO: `/v3/users/771cbd58af4f40afb427b7deeb9a9d3c` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:56.013216 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff9771-65e9-443f-8ea6-97ebd3ded0c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=771cbd58af4f40afb427b7deeb9a9d3c)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:56.015160 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff9771-65e9-443f-8ea6-97ebd3ded0c8 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:56.315411 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-80ff9771-65e9-443f-8ea6-97ebd3ded0c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '771cbd58af4f40afb427b7deeb9a9d3c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:56.322720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-80ff9771-65e9-443f-8ea6-97ebd3ded0c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '771cbd58af4f40afb427b7deeb9a9d3c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:56.409619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-80ff9771-65e9-443f-8ea6-97ebd3ded0c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '771cbd58af4f40afb427b7deeb9a9d3c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:56.410914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-80ff9771-65e9-443f-8ea6-97ebd3ded0c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '771cbd58af4f40afb427b7deeb9a9d3c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:56.433438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-80ff9771-65e9-443f-8ea6-97ebd3ded0c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '771cbd58af4f40afb427b7deeb9a9d3c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:56.440736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1298/2589] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:55 2022] DELETE /identity/v3/users/771cbd58af4f40afb427b7deeb9a9d3c => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:56.888309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:56.888309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:56.905302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fb59a-f614-4b4e-bb82-d93c217b0223 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:56.963165 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:56.963792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:56.964080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:56.964387 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a admin admin] PATH_INFO: `/v3/projects/3b3072ccd6144c7583c306ba00cc86a0` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:56.965495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b3072ccd6144c7583c306ba00cc86a0)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:56.967404 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:57.079020 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b3072ccd6144c7583c306ba00cc86a0', 'request_id': 'req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.085641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b3072ccd6144c7583c306ba00cc86a0', 'request_id': 'req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.161829 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b3072ccd6144c7583c306ba00cc86a0', 'request_id': 'req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.173586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a admin admin] The token cache is being invalidate because project 3b3072ccd6144c7583c306ba00cc86a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:57.174444 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.175099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-625da0c2-29ee-4b91-8fd2-ae2b5414f77a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.176623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1292/2590] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:56 2022] DELETE /identity/v3/projects/3b3072ccd6144c7583c306ba00cc86a0 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:57.181939 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b72749-facd-4652-8d01-801643c5359f None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.182421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b72749-facd-4652-8d01-801643c5359f None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.182546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b72749-facd-4652-8d01-801643c5359f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.201749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-d3b72749-facd-4652-8d01-801643c5359f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:57.202953 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d3b72749-facd-4652-8d01-801643c5359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:57.240640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:57.268051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1299/2591] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:57.277182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b965cc2-3e5e-4a11-9f31-1819f4601c8f None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.277493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b965cc2-3e5e-4a11-9f31-1819f4601c8f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.277771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b965cc2-3e5e-4a11-9f31-1819f4601c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.297095 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-1b965cc2-3e5e-4a11-9f31-1819f4601c8f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:57.298292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1b965cc2-3e5e-4a11-9f31-1819f4601c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:57.306547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1293/2592] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:57.310799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b72749-facd-4652-8d01-801643c5359f None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:57.322277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b72749-facd-4652-8d01-801643c5359f None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:57.327493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac47b21-3562-49de-8653-545461e31a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:57.328398 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac47b21-3562-49de-8653-545461e31a04 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.336908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac47b21-3562-49de-8653-545461e31a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.336908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac47b21-3562-49de-8653-545461e31a04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.336908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac47b21-3562-49de-8653-545461e31a04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:57.336908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac47b21-3562-49de-8653-545461e31a04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:57.336908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac47b21-3562-49de-8653-545461e31a04 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:57.343928 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1300/2593] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:57.349061 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b965cc2-3e5e-4a11-9f31-1819f4601c8f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:57.358343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b965cc2-3e5e-4a11-9f31-1819f4601c8f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:57.363038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806dde06-2013-4db5-87c0-ff5ebca05850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:57.363624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dde06-2013-4db5-87c0-ff5ebca05850 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.363899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dde06-2013-4db5-87c0-ff5ebca05850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.364185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dde06-2013-4db5-87c0-ff5ebca05850 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.364822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806dde06-2013-4db5-87c0-ff5ebca05850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:57.366748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806dde06-2013-4db5-87c0-ff5ebca05850 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:57.496257 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1294/2594] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:57 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:57.500597 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac47b21-3562-49de-8653-545461e31a04 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:57.510818 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac47b21-3562-49de-8653-545461e31a04 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:57.570688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aaa9da-fa1c-44f3-892a-a39925eef218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:57.571304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaa9da-fa1c-44f3-892a-a39925eef218 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.571614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaa9da-fa1c-44f3-892a-a39925eef218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.571901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaa9da-fa1c-44f3-892a-a39925eef218 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.572621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aaa9da-fa1c-44f3-892a-a39925eef218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:57.574622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aaa9da-fa1c-44f3-892a-a39925eef218 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:57.700631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1301/2595] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:57 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:57.702166 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806dde06-2013-4db5-87c0-ff5ebca05850 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:57.713351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806dde06-2013-4db5-87c0-ff5ebca05850 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:57.718294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f5252-cbcf-4c6d-926a-a99d557dd38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:57.720224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f5252-cbcf-4c6d-926a-a99d557dd38c admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.720224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f5252-cbcf-4c6d-926a-a99d557dd38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.720224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f5252-cbcf-4c6d-926a-a99d557dd38c admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.720565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f5252-cbcf-4c6d-926a-a99d557dd38c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:57.722353 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f5252-cbcf-4c6d-926a-a99d557dd38c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:57.734269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1295/2596] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:57.738581 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aaa9da-fa1c-44f3-892a-a39925eef218 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:57.749051 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aaa9da-fa1c-44f3-892a-a39925eef218 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:57.755663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0973d-b7dd-4b64-8232-762d0f33c95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:57.755987 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0973d-b7dd-4b64-8232-762d0f33c95f admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.755987 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0973d-b7dd-4b64-8232-762d0f33c95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.756293 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0973d-b7dd-4b64-8232-762d0f33c95f admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.756949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0973d-b7dd-4b64-8232-762d0f33c95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:57.758588 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0973d-b7dd-4b64-8232-762d0f33c95f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:57.773714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1302/2597] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:57.774897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f5252-cbcf-4c6d-926a-a99d557dd38c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:57.784963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f5252-cbcf-4c6d-926a-a99d557dd38c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:57.789775 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c14cae-dd2d-443e-a529-8ff397e4f108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:57.790425 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c14cae-dd2d-443e-a529-8ff397e4f108 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.790705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c14cae-dd2d-443e-a529-8ff397e4f108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.790981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c14cae-dd2d-443e-a529-8ff397e4f108 admin admin] PATH_INFO: `/v3/projects/81d4dc579fb0405c9955cf39cfc80184/users/74b1a3e6bfb846309de3a5eb01d60ac4/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.802677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:57.806765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c14cae-dd2d-443e-a529-8ff397e4f108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d4dc579fb0405c9955cf39cfc80184, user_id=74b1a3e6bfb846309de3a5eb01d60ac4, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:57.809689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c14cae-dd2d-443e-a529-8ff397e4f108 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:57.906692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1296/2598] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:57 2022] PUT /identity/v3/projects/81d4dc579fb0405c9955cf39cfc80184/users/74b1a3e6bfb846309de3a5eb01d60ac4/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:57.917043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7383b4-d7b3-4f9c-b31d-57a1adfb1f91 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.917390 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7383b4-d7b3-4f9c-b31d-57a1adfb1f91 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.917673 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7383b4-d7b3-4f9c-b31d-57a1adfb1f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.946389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-cc7383b4-d7b3-4f9c-b31d-57a1adfb1f91 None None] MFA Rules not processed for user `74b1a3e6bfb846309de3a5eb01d60ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:57.947575 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cc7383b4-d7b3-4f9c-b31d-57a1adfb1f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:58.014152 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1303/2599] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:58.019671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c14cae-dd2d-443e-a529-8ff397e4f108 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:58.092703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c14cae-dd2d-443e-a529-8ff397e4f108 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:58.097574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c686ec-3900-4f89-bb5d-7a871943b938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:58.098213 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c686ec-3900-4f89-bb5d-7a871943b938 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.098502 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c686ec-3900-4f89-bb5d-7a871943b938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.098769 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c686ec-3900-4f89-bb5d-7a871943b938 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.099431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c686ec-3900-4f89-bb5d-7a871943b938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:58.101384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c686ec-3900-4f89-bb5d-7a871943b938 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:58.219125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1297/2600] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:58 2022] POST /identity/v3/projects => generated 387 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:58.220292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7383b4-d7b3-4f9c-b31d-57a1adfb1f91 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:58.232062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7383b4-d7b3-4f9c-b31d-57a1adfb1f91 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:58.270792 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:58.302538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da3fcf-a66c-4379-8d7b-11ebb737a709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:58.303280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da3fcf-a66c-4379-8d7b-11ebb737a709 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.303570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da3fcf-a66c-4379-8d7b-11ebb737a709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.303846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da3fcf-a66c-4379-8d7b-11ebb737a709 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.304524 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da3fcf-a66c-4379-8d7b-11ebb737a709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:58.306455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da3fcf-a66c-4379-8d7b-11ebb737a709 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:58.500923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1304/2601] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:58 2022] POST /identity/v3/users => generated 300 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:58.505967 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c686ec-3900-4f89-bb5d-7a871943b938 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:58.518215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c686ec-3900-4f89-bb5d-7a871943b938 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:58.523064 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a486fb3-4978-462d-9b43-aa0d2da38bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:58.523660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a486fb3-4978-462d-9b43-aa0d2da38bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.523939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a486fb3-4978-462d-9b43-aa0d2da38bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.524235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a486fb3-4978-462d-9b43-aa0d2da38bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.524905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a486fb3-4978-462d-9b43-aa0d2da38bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:58.526492 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a486fb3-4978-462d-9b43-aa0d2da38bd3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:58.543942 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1298/2602] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:34:58.547321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da3fcf-a66c-4379-8d7b-11ebb737a709 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:58.557956 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da3fcf-a66c-4379-8d7b-11ebb737a709 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:58.562801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34415c-4403-481b-8334-8bca02a9b1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:58.563461 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34415c-4403-481b-8334-8bca02a9b1a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.563742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34415c-4403-481b-8334-8bca02a9b1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.564016 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34415c-4403-481b-8334-8bca02a9b1a1 admin admin] PATH_INFO: `/v3/projects/2c1e48ffff2d44cea575c22713cab85a/users/49aa9f2b89b04243beca90dad7dcaecc/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.581082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34415c-4403-481b-8334-8bca02a9b1a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c1e48ffff2d44cea575c22713cab85a, user_id=49aa9f2b89b04243beca90dad7dcaecc, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:58.583951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34415c-4403-481b-8334-8bca02a9b1a1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:58.752884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1305/2603] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:58 2022] PUT /identity/v3/projects/2c1e48ffff2d44cea575c22713cab85a/users/49aa9f2b89b04243beca90dad7dcaecc/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:58.757062 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a486fb3-4978-462d-9b43-aa0d2da38bd3 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:58.766984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a486fb3-4978-462d-9b43-aa0d2da38bd3 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:58.832470 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177db7a-af2a-4239-9f71-7fc1130e89bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:58.833197 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177db7a-af2a-4239-9f71-7fc1130e89bb admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.833484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177db7a-af2a-4239-9f71-7fc1130e89bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.833767 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177db7a-af2a-4239-9f71-7fc1130e89bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.834412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177db7a-af2a-4239-9f71-7fc1130e89bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:58.835997 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177db7a-af2a-4239-9f71-7fc1130e89bb admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:58.846867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1299/2604] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:58.851072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34415c-4403-481b-8334-8bca02a9b1a1 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:58.861852 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34415c-4403-481b-8334-8bca02a9b1a1 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:58.866915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d05ce6-668e-4e04-b290-4233fbaace6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:58.867565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d05ce6-668e-4e04-b290-4233fbaace6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.867848 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d05ce6-668e-4e04-b290-4233fbaace6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.868149 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d05ce6-668e-4e04-b290-4233fbaace6c admin admin] PATH_INFO: `/v3/projects/2c1e48ffff2d44cea575c22713cab85a/users/49aa9f2b89b04243beca90dad7dcaecc/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.870195 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d05ce6-668e-4e04-b290-4233fbaace6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c1e48ffff2d44cea575c22713cab85a, user_id=49aa9f2b89b04243beca90dad7dcaecc, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:58.872939 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d05ce6-668e-4e04-b290-4233fbaace6c admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:58.884847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1306/2605] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:58 2022] PUT /identity/v3/projects/2c1e48ffff2d44cea575c22713cab85a/users/49aa9f2b89b04243beca90dad7dcaecc/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:58.895248 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4684093-c538-4126-9e3a-2da78a690770 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.895543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4684093-c538-4126-9e3a-2da78a690770 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.895826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4684093-c538-4126-9e3a-2da78a690770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.924770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-a4684093-c538-4126-9e3a-2da78a690770 None None] MFA Rules not processed for user `49aa9f2b89b04243beca90dad7dcaecc`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:58.926031 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a4684093-c538-4126-9e3a-2da78a690770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:58.980838 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:59.007861 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1300/2606] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:34:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:59.015387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d05ce6-668e-4e04-b290-4233fbaace6c admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:59.017800 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-44d05ce6-668e-4e04-b290-4233fbaace6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:59.087072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d05ce6-668e-4e04-b290-4233fbaace6c admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:59.091495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4522bc-6a57-4cd1-ba9f-3425aa25a79e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:59.092115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4522bc-6a57-4cd1-ba9f-3425aa25a79e service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:59.092441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4522bc-6a57-4cd1-ba9f-3425aa25a79e service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:59.092787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4522bc-6a57-4cd1-ba9f-3425aa25a79e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:59.104098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4522bc-6a57-4cd1-ba9f-3425aa25a79e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:59.108469 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4522bc-6a57-4cd1-ba9f-3425aa25a79e service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:59.115998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1307/2607] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:59 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:00.586452 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4684093-c538-4126-9e3a-2da78a690770 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:00.732079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4684093-c538-4126-9e3a-2da78a690770 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:00.737060 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:00.737757 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:00.738044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:00.739933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] PATH_INFO: `/v3/users/74b1a3e6bfb846309de3a5eb01d60ac4` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:00.739933 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b1a3e6bfb846309de3a5eb01d60ac4)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:00.742340 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:01.017278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b1a3e6bfb846309de3a5eb01d60ac4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:01.017946 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b1a3e6bfb846309de3a5eb01d60ac4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:01.025423 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b1a3e6bfb846309de3a5eb01d60ac4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:01.116937 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b1a3e6bfb846309de3a5eb01d60ac4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:01.117641 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b1a3e6bfb846309de3a5eb01d60ac4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:01.135784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1301/2608] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:35:00 2022] DELETE /identity/v3/users/74b1a3e6bfb846309de3a5eb01d60ac4 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:01.140223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb077b-df72-4826-8b54-0ae503a71f42 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:01.140713 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb077b-df72-4826-8b54-0ae503a71f42 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:01.140844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb077b-df72-4826-8b54-0ae503a71f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:01.194068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3dcb077b-df72-4826-8b54-0ae503a71f42 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:01.196130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3dcb077b-df72-4826-8b54-0ae503a71f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:01.263153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1308/2609] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:01.273693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:01.275079 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:01.299667 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:01.349527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce387a-2b4c-4e43-9908-26f1634673e8 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:01.353174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a82ce-add5-45a7-95e7-8f69d4a4a88d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:01.353766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a82ce-add5-45a7-95e7-8f69d4a4a88d service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:01.354053 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a82ce-add5-45a7-95e7-8f69d4a4a88d service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:01.354315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a82ce-add5-45a7-95e7-8f69d4a4a88d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:01.362337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb077b-df72-4826-8b54-0ae503a71f42 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:01.363600 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a82ce-add5-45a7-95e7-8f69d4a4a88d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:01.365187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a82ce-add5-45a7-95e7-8f69d4a4a88d service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:01.366875 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:01.372557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb077b-df72-4826-8b54-0ae503a71f42 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:01.375041 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1302/2610] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:01.377032 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a00271-65ac-4287-8375-a1cd795e8dba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:01.377633 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a00271-65ac-4287-8375-a1cd795e8dba service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:01.377909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a00271-65ac-4287-8375-a1cd795e8dba service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:01.378175 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a00271-65ac-4287-8375-a1cd795e8dba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:01.380240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-56a00271-65ac-4287-8375-a1cd795e8dba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:01.443638 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a00271-65ac-4287-8375-a1cd795e8dba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:01.445143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a00271-65ac-4287-8375-a1cd795e8dba service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:01.454325 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1309/2611] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:35:01.988105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a82ce-add5-45a7-95e7-8f69d4a4a88d service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:01.989605 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1f3a82ce-add5-45a7-95e7-8f69d4a4a88d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:02.004876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a82ce-add5-45a7-95e7-8f69d4a4a88d service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:02.010232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:02.010904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:02.011186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:02.011457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] PATH_INFO: `/v3/projects/81d4dc579fb0405c9955cf39cfc80184` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:02.012611 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81d4dc579fb0405c9955cf39cfc80184)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:02.014505 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:02.170773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81d4dc579fb0405c9955cf39cfc80184', 'request_id': 'req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:02.177386 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81d4dc579fb0405c9955cf39cfc80184', 'request_id': 'req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:02.247528 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81d4dc579fb0405c9955cf39cfc80184', 'request_id': 'req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:02.260681 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] The token cache is being invalidate because project 81d4dc579fb0405c9955cf39cfc80184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:35:02.268600 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:02.274753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:02.276477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1303/2612] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:35:01 2022] DELETE /identity/v3/projects/81d4dc579fb0405c9955cf39cfc80184 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:02.280997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a00271-65ac-4287-8375-a1cd795e8dba service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:02.292531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a00271-65ac-4287-8375-a1cd795e8dba service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:02.355668 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cee07-672f-4f31-a721-7360b3e20d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:02.356233 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cee07-672f-4f31-a721-7360b3e20d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:02.356519 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cee07-672f-4f31-a721-7360b3e20d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:02.356792 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cee07-672f-4f31-a721-7360b3e20d8d admin admin] PATH_INFO: `/v3/users/49aa9f2b89b04243beca90dad7dcaecc` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:02.357930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cee07-672f-4f31-a721-7360b3e20d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=49aa9f2b89b04243beca90dad7dcaecc)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:02.359953 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cee07-672f-4f31-a721-7360b3e20d8d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:02.631155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-491cee07-672f-4f31-a721-7360b3e20d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49aa9f2b89b04243beca90dad7dcaecc'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:02.639940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-491cee07-672f-4f31-a721-7360b3e20d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49aa9f2b89b04243beca90dad7dcaecc'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:02.758003 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-491cee07-672f-4f31-a721-7360b3e20d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49aa9f2b89b04243beca90dad7dcaecc'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:02.758883 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-491cee07-672f-4f31-a721-7360b3e20d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49aa9f2b89b04243beca90dad7dcaecc'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:02.780665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-491cee07-672f-4f31-a721-7360b3e20d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49aa9f2b89b04243beca90dad7dcaecc'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:02.801979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1310/2613] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:35:02 2022] DELETE /identity/v3/users/49aa9f2b89b04243beca90dad7dcaecc => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:03.383676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:03.385308 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:03.405264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcba27-2471-4456-8a66-e5ccf06f5ea8 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:03.477780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:03.478420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:03.478698 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:03.478972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab admin admin] PATH_INFO: `/v3/projects/2c1e48ffff2d44cea575c22713cab85a` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:03.480119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c1e48ffff2d44cea575c22713cab85a)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:03.482120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:03.591692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:03.603630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c1e48ffff2d44cea575c22713cab85a', 'request_id': 'req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:03.613092 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c1e48ffff2d44cea575c22713cab85a', 'request_id': 'req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:03.734785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c1e48ffff2d44cea575c22713cab85a', 'request_id': 'req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:03.747324 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab admin admin] The token cache is being invalidate because project 2c1e48ffff2d44cea575c22713cab85a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:35:03.747744 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:03.748431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-87a2a2ff-2f57-483e-bd80-6e5ed59868ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:03.750036 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1304/2614] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:35:03 2022] DELETE /identity/v3/projects/2c1e48ffff2d44cea575c22713cab85a => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:03.755398 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf86ea0-e763-4db9-9c62-fe803d0e0bbe None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:03.755850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf86ea0-e763-4db9-9c62-fe803d0e0bbe None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:03.755945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf86ea0-e763-4db9-9c62-fe803d0e0bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:03.783214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:03.803783 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-7cf86ea0-e763-4db9-9c62-fe803d0e0bbe None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:03.804996 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7cf86ea0-e763-4db9-9c62-fe803d0e0bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:03.863078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1311/2615] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:03.870744 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e56cd-6934-4f30-8bf9-c4505d5be158 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:03.871024 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e56cd-6934-4f30-8bf9-c4505d5be158 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:03.871296 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e56cd-6934-4f30-8bf9-c4505d5be158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:03.893091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-639e56cd-6934-4f30-8bf9-c4505d5be158 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:03.894262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-639e56cd-6934-4f30-8bf9-c4505d5be158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:03.902590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1305/2616] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:03.906793 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf86ea0-e763-4db9-9c62-fe803d0e0bbe None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:03.916995 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf86ea0-e763-4db9-9c62-fe803d0e0bbe None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:03.922437 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d274e-d733-465e-abe1-8fb92269b5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:03.923069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d274e-d733-465e-abe1-8fb92269b5ac admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:03.924099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d274e-d733-465e-abe1-8fb92269b5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:03.924445 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d274e-d733-465e-abe1-8fb92269b5ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:03.925095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d274e-d733-465e-abe1-8fb92269b5ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:35:03.925404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d274e-d733-465e-abe1-8fb92269b5ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:03.927046 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d274e-d733-465e-abe1-8fb92269b5ac admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:03.938538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1312/2617] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:35:03.943450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e56cd-6934-4f30-8bf9-c4505d5be158 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:03.952903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e56cd-6934-4f30-8bf9-c4505d5be158 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:03.958327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bd8e4-246f-4365-a437-c3399375d970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:03.958932 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bd8e4-246f-4365-a437-c3399375d970 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:03.959240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bd8e4-246f-4365-a437-c3399375d970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:03.959514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bd8e4-246f-4365-a437-c3399375d970 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:03.960159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bd8e4-246f-4365-a437-c3399375d970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:03.962086 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bd8e4-246f-4365-a437-c3399375d970 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:04.039008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1306/2618] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:35:03 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:04.043158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d274e-d733-465e-abe1-8fb92269b5ac admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:04.053649 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d274e-d733-465e-abe1-8fb92269b5ac admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:04.111891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300a242-edb9-4732-9847-bd166ccc1024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:04.112563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300a242-edb9-4732-9847-bd166ccc1024 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:04.112863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300a242-edb9-4732-9847-bd166ccc1024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:04.113154 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300a242-edb9-4732-9847-bd166ccc1024 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:04.113809 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300a242-edb9-4732-9847-bd166ccc1024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:04.115923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300a242-edb9-4732-9847-bd166ccc1024 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:04.208237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1313/2619] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:35:04 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:04.212339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bd8e4-246f-4365-a437-c3399375d970 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:04.222713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bd8e4-246f-4365-a437-c3399375d970 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:04.227363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a18163-828d-4fbc-9f16-ccb53f2408b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:04.228010 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a18163-828d-4fbc-9f16-ccb53f2408b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:04.228257 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a18163-828d-4fbc-9f16-ccb53f2408b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:04.228601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a18163-828d-4fbc-9f16-ccb53f2408b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:04.229159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a18163-828d-4fbc-9f16-ccb53f2408b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:04.230742 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a18163-828d-4fbc-9f16-ccb53f2408b1 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:04.242267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1307/2620] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:04.246119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300a242-edb9-4732-9847-bd166ccc1024 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:04.257072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300a242-edb9-4732-9847-bd166ccc1024 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:04.261879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80806e6-ada5-41f9-9fd3-be497f1ab201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:04.262499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80806e6-ada5-41f9-9fd3-be497f1ab201 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:04.262762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80806e6-ada5-41f9-9fd3-be497f1ab201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:04.263033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80806e6-ada5-41f9-9fd3-be497f1ab201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:04.263676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80806e6-ada5-41f9-9fd3-be497f1ab201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:04.265296 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80806e6-ada5-41f9-9fd3-be497f1ab201 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:04.276605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1314/2621] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:04.280927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a18163-828d-4fbc-9f16-ccb53f2408b1 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:04.290791 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a18163-828d-4fbc-9f16-ccb53f2408b1 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:04.296058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310c780-c8ce-47d0-9c84-687366d9fbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:04.297366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310c780-c8ce-47d0-9c84-687366d9fbe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:04.297366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310c780-c8ce-47d0-9c84-687366d9fbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:04.297366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310c780-c8ce-47d0-9c84-687366d9fbe2 admin admin] PATH_INFO: `/v3/projects/1dbb48bab4844d9387d7cb1518b75d68/users/b0f0f6f231434cacad75c796fd46b59b/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:04.313044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310c780-c8ce-47d0-9c84-687366d9fbe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dbb48bab4844d9387d7cb1518b75d68, user_id=b0f0f6f231434cacad75c796fd46b59b, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:04.316651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310c780-c8ce-47d0-9c84-687366d9fbe2 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:04.397796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1308/2622] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:35:04 2022] PUT /identity/v3/projects/1dbb48bab4844d9387d7cb1518b75d68/users/b0f0f6f231434cacad75c796fd46b59b/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:04.408673 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7987be-9634-41b9-8d0d-8628f3f10c6f None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:04.408956 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7987be-9634-41b9-8d0d-8628f3f10c6f None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:04.409250 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7987be-9634-41b9-8d0d-8628f3f10c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:04.437930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-ff7987be-9634-41b9-8d0d-8628f3f10c6f None None] MFA Rules not processed for user `b0f0f6f231434cacad75c796fd46b59b`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:04.439694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-ff7987be-9634-41b9-8d0d-8628f3f10c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:04.488385 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:04.505018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1315/2623] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:04.509563 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310c780-c8ce-47d0-9c84-687366d9fbe2 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:04.573454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310c780-c8ce-47d0-9c84-687366d9fbe2 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:04.578529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00f0a6-b0f9-4163-b323-b229eceb25d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:04.579146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00f0a6-b0f9-4163-b323-b229eceb25d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:04.579431 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00f0a6-b0f9-4163-b323-b229eceb25d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:04.579748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00f0a6-b0f9-4163-b323-b229eceb25d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:04.580464 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00f0a6-b0f9-4163-b323-b229eceb25d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:04.582403 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00f0a6-b0f9-4163-b323-b229eceb25d0 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:04.677229 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1309/2624] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:35:04 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:04.678529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7987be-9634-41b9-8d0d-8628f3f10c6f None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:04.690126 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7987be-9634-41b9-8d0d-8628f3f10c6f None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:04.749376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b720aa-a8f3-4c58-b1d1-c9922442dc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:04.750034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b720aa-a8f3-4c58-b1d1-c9922442dc43 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:04.750305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b720aa-a8f3-4c58-b1d1-c9922442dc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:04.750577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b720aa-a8f3-4c58-b1d1-c9922442dc43 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:04.751206 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b720aa-a8f3-4c58-b1d1-c9922442dc43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:04.753441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b720aa-a8f3-4c58-b1d1-c9922442dc43 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:04.893755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1316/2625] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:35:04 2022] POST /identity/v3/users => generated 299 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:35:04.895988 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00f0a6-b0f9-4163-b323-b229eceb25d0 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:04.911117 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00f0a6-b0f9-4163-b323-b229eceb25d0 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:04.922705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009613a-49bf-42d4-9912-b18a9f7ac696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:04.923249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009613a-49bf-42d4-9912-b18a9f7ac696 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:04.924241 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009613a-49bf-42d4-9912-b18a9f7ac696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:04.924523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009613a-49bf-42d4-9912-b18a9f7ac696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:04.925172 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009613a-49bf-42d4-9912-b18a9f7ac696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:04.926787 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009613a-49bf-42d4-9912-b18a9f7ac696 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:04.944341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1310/2626] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:04.949102 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b720aa-a8f3-4c58-b1d1-c9922442dc43 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:04.965202 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b720aa-a8f3-4c58-b1d1-c9922442dc43 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:04.974452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482da46d-e973-4425-acdf-75ba0b685ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:04.975131 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482da46d-e973-4425-acdf-75ba0b685ef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:04.975410 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482da46d-e973-4425-acdf-75ba0b685ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:04.975723 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482da46d-e973-4425-acdf-75ba0b685ef2 admin admin] PATH_INFO: `/v3/projects/5ab4dbe57f1d4d289e95ab2bfdc7a262/users/adec15bc0552461e93ede24b1b7b3b03/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:04.994555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482da46d-e973-4425-acdf-75ba0b685ef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab4dbe57f1d4d289e95ab2bfdc7a262, user_id=adec15bc0552461e93ede24b1b7b3b03, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:04.997276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482da46d-e973-4425-acdf-75ba0b685ef2 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:05.123500 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1317/2627] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:35:04 2022] PUT /identity/v3/projects/5ab4dbe57f1d4d289e95ab2bfdc7a262/users/adec15bc0552461e93ede24b1b7b3b03/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:05.128075 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009613a-49bf-42d4-9912-b18a9f7ac696 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:05.138137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009613a-49bf-42d4-9912-b18a9f7ac696 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:05.175232 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:05.205437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad8cf4-fd53-41c1-b8ed-b761d50efd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:05.206062 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad8cf4-fd53-41c1-b8ed-b761d50efd88 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:05.206525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad8cf4-fd53-41c1-b8ed-b761d50efd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:05.206711 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad8cf4-fd53-41c1-b8ed-b761d50efd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:05.207364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad8cf4-fd53-41c1-b8ed-b761d50efd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:05.209105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad8cf4-fd53-41c1-b8ed-b761d50efd88 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:05.220724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1311/2628] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:05.224792 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482da46d-e973-4425-acdf-75ba0b685ef2 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:05.235287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482da46d-e973-4425-acdf-75ba0b685ef2 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:05.240043 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045810a0-54f2-4189-8d63-97cace73b6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:05.240797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045810a0-54f2-4189-8d63-97cace73b6c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:05.241077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045810a0-54f2-4189-8d63-97cace73b6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:05.241374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045810a0-54f2-4189-8d63-97cace73b6c5 admin admin] PATH_INFO: `/v3/projects/5ab4dbe57f1d4d289e95ab2bfdc7a262/users/adec15bc0552461e93ede24b1b7b3b03/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:05.243380 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045810a0-54f2-4189-8d63-97cace73b6c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab4dbe57f1d4d289e95ab2bfdc7a262, user_id=adec15bc0552461e93ede24b1b7b3b03, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:05.246135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045810a0-54f2-4189-8d63-97cace73b6c5 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:05.259147 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1318/2629] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:35:05 2022] PUT /identity/v3/projects/5ab4dbe57f1d4d289e95ab2bfdc7a262/users/adec15bc0552461e93ede24b1b7b3b03/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:05.271092 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c1fad-a8e0-4040-908e-ac069eb28458 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:05.271377 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c1fad-a8e0-4040-908e-ac069eb28458 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:05.271753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c1fad-a8e0-4040-908e-ac069eb28458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:05.300038 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f83c1fad-a8e0-4040-908e-ac069eb28458 None None] MFA Rules not processed for user `adec15bc0552461e93ede24b1b7b3b03`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:05.301302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f83c1fad-a8e0-4040-908e-ac069eb28458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:05.370885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1312/2630] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:05.381929 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045810a0-54f2-4189-8d63-97cace73b6c5 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:05.383648 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-045810a0-54f2-4189-8d63-97cace73b6c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:05.454252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045810a0-54f2-4189-8d63-97cace73b6c5 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:05.459075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f707b9a-bade-4c92-9996-491510022191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:05.459736 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f707b9a-bade-4c92-9996-491510022191 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:05.460017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f707b9a-bade-4c92-9996-491510022191 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:05.460327 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f707b9a-bade-4c92-9996-491510022191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:05.531695 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f707b9a-bade-4c92-9996-491510022191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:05.533886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f707b9a-bade-4c92-9996-491510022191 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:05.538666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:05.548254 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1319/2631] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:09.261647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c1fad-a8e0-4040-908e-ac069eb28458 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:09.263686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f83c1fad-a8e0-4040-908e-ac069eb28458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:09.321525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c1fad-a8e0-4040-908e-ac069eb28458 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:09.325315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1c1ff-6eea-49ba-94ee-80ffd3165b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:09.325921 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1c1ff-6eea-49ba-94ee-80ffd3165b98 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:09.326204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1c1ff-6eea-49ba-94ee-80ffd3165b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:09.326478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1c1ff-6eea-49ba-94ee-80ffd3165b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:09.328249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e6e1c1ff-6eea-49ba-94ee-80ffd3165b98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:09.395261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1c1ff-6eea-49ba-94ee-80ffd3165b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:09.396842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1c1ff-6eea-49ba-94ee-80ffd3165b98 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:09.406879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1313/2632] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:35:11.431619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f707b9a-bade-4c92-9996-491510022191 service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:11.440694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f707b9a-bade-4c92-9996-491510022191 service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:11.444710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360be080-bb64-4809-98a6-9c4e40bd28d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:11.446395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360be080-bb64-4809-98a6-9c4e40bd28d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:11.446395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360be080-bb64-4809-98a6-9c4e40bd28d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:11.446395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360be080-bb64-4809-98a6-9c4e40bd28d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:11.456697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360be080-bb64-4809-98a6-9c4e40bd28d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:11.458465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360be080-bb64-4809-98a6-9c4e40bd28d7 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:11.468084 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1320/2633] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:17.701837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1c1ff-6eea-49ba-94ee-80ffd3165b98 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:17.703374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e6e1c1ff-6eea-49ba-94ee-80ffd3165b98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:17.711897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:17.716459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1c1ff-6eea-49ba-94ee-80ffd3165b98 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:17.721180 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a4b96-07e0-49fd-a0a4-1659357188e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:17.721801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a4b96-07e0-49fd-a0a4-1659357188e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:17.722084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a4b96-07e0-49fd-a0a4-1659357188e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:17.722361 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a4b96-07e0-49fd-a0a4-1659357188e3 admin admin] PATH_INFO: `/v3/users/95d668bfaf71417baac3c4336c33ed9f` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:17.723478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a4b96-07e0-49fd-a0a4-1659357188e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95d668bfaf71417baac3c4336c33ed9f)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:17.725428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a4b96-07e0-49fd-a0a4-1659357188e3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:17.913316 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-083a4b96-07e0-49fd-a0a4-1659357188e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95d668bfaf71417baac3c4336c33ed9f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:17.914078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-083a4b96-07e0-49fd-a0a4-1659357188e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95d668bfaf71417baac3c4336c33ed9f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:17.921327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-083a4b96-07e0-49fd-a0a4-1659357188e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95d668bfaf71417baac3c4336c33ed9f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:17.942415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360be080-bb64-4809-98a6-9c4e40bd28d7 service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:17.952747 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360be080-bb64-4809-98a6-9c4e40bd28d7 service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:17.957452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:17.958082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:17.958361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:17.958629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] PATH_INFO: `/v3/users/b0f0f6f231434cacad75c796fd46b59b` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:17.959758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0f0f6f231434cacad75c796fd46b59b)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:17.961863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:18.031078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-083a4b96-07e0-49fd-a0a4-1659357188e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95d668bfaf71417baac3c4336c33ed9f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:18.031705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-083a4b96-07e0-49fd-a0a4-1659357188e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95d668bfaf71417baac3c4336c33ed9f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:18.049712 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1314/2634] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:35:17 2022] DELETE /identity/v3/users/95d668bfaf71417baac3c4336c33ed9f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:18.201818 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0f0f6f231434cacad75c796fd46b59b'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:18.209700 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0f0f6f231434cacad75c796fd46b59b'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:18.293964 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0f0f6f231434cacad75c796fd46b59b'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:18.294583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0f0f6f231434cacad75c796fd46b59b'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:18.307850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0f0f6f231434cacad75c796fd46b59b'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:18.314368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1321/2635] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:35:17 2022] DELETE /identity/v3/users/b0f0f6f231434cacad75c796fd46b59b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:35:18.318874 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90647042-8b3b-4a22-8094-7b86637d8aee None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:18.319953 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90647042-8b3b-4a22-8094-7b86637d8aee None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:18.319953 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90647042-8b3b-4a22-8094-7b86637d8aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:18.339801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-90647042-8b3b-4a22-8094-7b86637d8aee None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:18.341057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-90647042-8b3b-4a22-8094-7b86637d8aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:18.348087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:18.353594 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:18.405262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1315/2636] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:18.415470 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90647042-8b3b-4a22-8094-7b86637d8aee None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:18.416951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-90647042-8b3b-4a22-8094-7b86637d8aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:18.438499 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6dc626-c8ab-4887-a5b3-0eeadb38de43 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:18.443917 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:18.444299 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:18.444580 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:18.444879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] PATH_INFO: `/v3/projects/8c28205e89c1441bbb3f1d68ac8893e4` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:18.446042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c28205e89c1441bbb3f1d68ac8893e4)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:18.448362 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:18.476278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90647042-8b3b-4a22-8094-7b86637d8aee None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:18.479899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b329a-8ec5-4586-9cea-93b6381c290d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:18.480672 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b329a-8ec5-4586-9cea-93b6381c290d service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:18.481003 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b329a-8ec5-4586-9cea-93b6381c290d service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:18.481264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b329a-8ec5-4586-9cea-93b6381c290d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:18.490509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b329a-8ec5-4586-9cea-93b6381c290d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:18.491957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b329a-8ec5-4586-9cea-93b6381c290d service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:18.501524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1316/2637] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:35:18.565319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c28205e89c1441bbb3f1d68ac8893e4', 'request_id': 'req-bcedaa64-26be-4d4a-97a0-b91a808a64d7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:18.572123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c28205e89c1441bbb3f1d68ac8893e4', 'request_id': 'req-bcedaa64-26be-4d4a-97a0-b91a808a64d7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:18.576629 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:18.579052 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c28205e89c1441bbb3f1d68ac8893e4', 'request_id': 'req-bcedaa64-26be-4d4a-97a0-b91a808a64d7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:18.671575 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] The token cache is being invalidate because project 8c28205e89c1441bbb3f1d68ac8893e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:35:18.671916 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:18.672681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:18.674859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1322/2638] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:35:18 2022] DELETE /identity/v3/projects/8c28205e89c1441bbb3f1d68ac8893e4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:18.679088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b329a-8ec5-4586-9cea-93b6381c290d service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:18.689560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b329a-8ec5-4586-9cea-93b6381c290d service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:18.706712 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:18.747185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:18.747834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:18.748119 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:18.748447 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] PATH_INFO: `/v3/users/cee743990ef84e1594242c1c51bcc13e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:18.749805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee743990ef84e1594242c1c51bcc13e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:18.752552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:18.969514 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:18.973708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:18.984433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedaa64-26be-4d4a-97a0-b91a808a64d7 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:18.992819 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:19.011822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:19.011822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:19.011822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] PATH_INFO: `/v3/projects/1dbb48bab4844d9387d7cb1518b75d68` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:19.011822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dbb48bab4844d9387d7cb1518b75d68)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:19.011822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:19.012856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee743990ef84e1594242c1c51bcc13e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.012856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee743990ef84e1594242c1c51bcc13e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.027325 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee743990ef84e1594242c1c51bcc13e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.139609 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee743990ef84e1594242c1c51bcc13e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.140254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee743990ef84e1594242c1c51bcc13e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.157123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dbb48bab4844d9387d7cb1518b75d68', 'request_id': 'req-e6044a13-0af8-4ae1-9c15-8a4d641893e1'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.158287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1317/2639] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:35:18 2022] DELETE /identity/v3/users/cee743990ef84e1594242c1c51bcc13e => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:19.164004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dbb48bab4844d9387d7cb1518b75d68', 'request_id': 'req-e6044a13-0af8-4ae1-9c15-8a4d641893e1'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.170089 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dbb48bab4844d9387d7cb1518b75d68', 'request_id': 'req-e6044a13-0af8-4ae1-9c15-8a4d641893e1'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.264765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] The token cache is being invalidate because project 1dbb48bab4844d9387d7cb1518b75d68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:35:19.265064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.265709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.267701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1323/2640] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:35:18 2022] DELETE /identity/v3/projects/1dbb48bab4844d9387d7cb1518b75d68 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:35:19.271714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:19.273482 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:19.341574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a608fb-dce1-4891-b4d0-9a8143af1b47 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:19.346803 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:19.347435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:19.347720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:19.348028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] PATH_INFO: `/v3/users/adec15bc0552461e93ede24b1b7b3b03` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:19.349217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adec15bc0552461e93ede24b1b7b3b03)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:19.351158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:19.438715 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:19.440395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:19.506259 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6044a13-0af8-4ae1-9c15-8a4d641893e1 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:19.511054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9276419-83c4-4f14-a936-1ebdac17a4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:19.511557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9276419-83c4-4f14-a936-1ebdac17a4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:19.511839 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9276419-83c4-4f14-a936-1ebdac17a4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:19.512115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9276419-83c4-4f14-a936-1ebdac17a4a6 admin admin] PATH_INFO: `/v3/projects/32871c359a34494cbd142bde9abfaa55` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:19.513280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9276419-83c4-4f14-a936-1ebdac17a4a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32871c359a34494cbd142bde9abfaa55)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:19.515186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9276419-83c4-4f14-a936-1ebdac17a4a6 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:19.599555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adec15bc0552461e93ede24b1b7b3b03'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.600228 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adec15bc0552461e93ede24b1b7b3b03'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.607077 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adec15bc0552461e93ede24b1b7b3b03'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.618859 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:19.673854 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d9276419-83c4-4f14-a936-1ebdac17a4a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32871c359a34494cbd142bde9abfaa55', 'request_id': 'req-d9276419-83c4-4f14-a936-1ebdac17a4a6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.684459 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d9276419-83c4-4f14-a936-1ebdac17a4a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32871c359a34494cbd142bde9abfaa55', 'request_id': 'req-d9276419-83c4-4f14-a936-1ebdac17a4a6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.690891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d9276419-83c4-4f14-a936-1ebdac17a4a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32871c359a34494cbd142bde9abfaa55', 'request_id': 'req-d9276419-83c4-4f14-a936-1ebdac17a4a6'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.700365 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adec15bc0552461e93ede24b1b7b3b03'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.700991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adec15bc0552461e93ede24b1b7b3b03'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.720742 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1318/2641] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:35:19 2022] DELETE /identity/v3/users/adec15bc0552461e93ede24b1b7b3b03 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:19.826976 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:19.830925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d9276419-83c4-4f14-a936-1ebdac17a4a6 admin admin] The token cache is being invalidate because project 32871c359a34494cbd142bde9abfaa55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:35:19.831221 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d9276419-83c4-4f14-a936-1ebdac17a4a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.831849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d9276419-83c4-4f14-a936-1ebdac17a4a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:19.833365 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1324/2642] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:35:19 2022] DELETE /identity/v3/projects/32871c359a34494cbd142bde9abfaa55 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:35:20.210831 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:20.212417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:20.282771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5be4c3-c68b-4d6d-b9cd-6c504fe446d7 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:20.287484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812d1ed-0cc4-48bb-b09a-ac393241c720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:20.288102 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812d1ed-0cc4-48bb-b09a-ac393241c720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:20.288406 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812d1ed-0cc4-48bb-b09a-ac393241c720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:20.288676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812d1ed-0cc4-48bb-b09a-ac393241c720 admin admin] PATH_INFO: `/v3/projects/5ab4dbe57f1d4d289e95ab2bfdc7a262` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:20.289801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812d1ed-0cc4-48bb-b09a-ac393241c720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab4dbe57f1d4d289e95ab2bfdc7a262)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:20.291707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812d1ed-0cc4-48bb-b09a-ac393241c720 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:20.416935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4812d1ed-0cc4-48bb-b09a-ac393241c720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab4dbe57f1d4d289e95ab2bfdc7a262', 'request_id': 'req-4812d1ed-0cc4-48bb-b09a-ac393241c720'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:20.423485 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4812d1ed-0cc4-48bb-b09a-ac393241c720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab4dbe57f1d4d289e95ab2bfdc7a262', 'request_id': 'req-4812d1ed-0cc4-48bb-b09a-ac393241c720'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:20.540323 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4812d1ed-0cc4-48bb-b09a-ac393241c720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab4dbe57f1d4d289e95ab2bfdc7a262', 'request_id': 'req-4812d1ed-0cc4-48bb-b09a-ac393241c720'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:20.552084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4812d1ed-0cc4-48bb-b09a-ac393241c720 admin admin] The token cache is being invalidate because project 5ab4dbe57f1d4d289e95ab2bfdc7a262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:35:20.552399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4812d1ed-0cc4-48bb-b09a-ac393241c720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:20.553122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-4812d1ed-0cc4-48bb-b09a-ac393241c720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:20.554645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1319/2643] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:35:20 2022] DELETE /identity/v3/projects/5ab4dbe57f1d4d289e95ab2bfdc7a262 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:20.560725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf73f8-fcd3-403d-9c19-c13f87b97c90 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:20.561198 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf73f8-fcd3-403d-9c19-c13f87b97c90 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:20.561330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf73f8-fcd3-403d-9c19-c13f87b97c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:20.582330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-27bf73f8-fcd3-403d-9c19-c13f87b97c90 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:20.584100 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-27bf73f8-fcd3-403d-9c19-c13f87b97c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:20.644801 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1325/2644] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:20.653417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f5580-58aa-4651-acbb-1be3579de990 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:20.653705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f5580-58aa-4651-acbb-1be3579de990 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:20.653985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f5580-58aa-4651-acbb-1be3579de990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:20.674664 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-e30f5580-58aa-4651-acbb-1be3579de990 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:20.675956 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e30f5580-58aa-4651-acbb-1be3579de990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:20.685158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1320/2645] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:20.695192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf73f8-fcd3-403d-9c19-c13f87b97c90 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:20.706094 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf73f8-fcd3-403d-9c19-c13f87b97c90 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:20.710799 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0461212-1e04-4d2e-9d3f-2e9a17c583b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:20.711401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0461212-1e04-4d2e-9d3f-2e9a17c583b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:20.711683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0461212-1e04-4d2e-9d3f-2e9a17c583b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:20.711960 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0461212-1e04-4d2e-9d3f-2e9a17c583b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:20.712672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0461212-1e04-4d2e-9d3f-2e9a17c583b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:35:20.712982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0461212-1e04-4d2e-9d3f-2e9a17c583b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:20.714716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0461212-1e04-4d2e-9d3f-2e9a17c583b1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:20.727504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1326/2646] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:35:20.735689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f5580-58aa-4651-acbb-1be3579de990 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:20.747574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f5580-58aa-4651-acbb-1be3579de990 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:20.752428 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bad5a1-45c1-4676-8405-ff872bfb9e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:20.753032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bad5a1-45c1-4676-8405-ff872bfb9e53 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:20.753314 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bad5a1-45c1-4676-8405-ff872bfb9e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:20.753588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bad5a1-45c1-4676-8405-ff872bfb9e53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:20.754272 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bad5a1-45c1-4676-8405-ff872bfb9e53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:20.756286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bad5a1-45c1-4676-8405-ff872bfb9e53 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:20.866754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1321/2647] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:35:20 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:20.870962 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0461212-1e04-4d2e-9d3f-2e9a17c583b1 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:20.881564 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0461212-1e04-4d2e-9d3f-2e9a17c583b1 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:20.939138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eea9a1-3aae-477b-ab1f-b4c4fafce376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:20.939741 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eea9a1-3aae-477b-ab1f-b4c4fafce376 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:20.940022 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eea9a1-3aae-477b-ab1f-b4c4fafce376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:20.940441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eea9a1-3aae-477b-ab1f-b4c4fafce376 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:20.941078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eea9a1-3aae-477b-ab1f-b4c4fafce376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:20.943114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eea9a1-3aae-477b-ab1f-b4c4fafce376 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:21.048946 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1327/2648] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:35:20 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:21.053055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bad5a1-45c1-4676-8405-ff872bfb9e53 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:21.063312 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bad5a1-45c1-4676-8405-ff872bfb9e53 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:21.068648 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa16e40-42d2-4dd6-ac9b-30ef6a6ea0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:21.069241 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa16e40-42d2-4dd6-ac9b-30ef6a6ea0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:21.069521 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa16e40-42d2-4dd6-ac9b-30ef6a6ea0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:21.069797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa16e40-42d2-4dd6-ac9b-30ef6a6ea0b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:21.070436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa16e40-42d2-4dd6-ac9b-30ef6a6ea0b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:21.072022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa16e40-42d2-4dd6-ac9b-30ef6a6ea0b0 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:21.082897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1322/2649] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:21.086844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eea9a1-3aae-477b-ab1f-b4c4fafce376 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:21.096780 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eea9a1-3aae-477b-ab1f-b4c4fafce376 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:21.101353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb14424-2ab5-4d3d-b919-0affea614482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:21.101934 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb14424-2ab5-4d3d-b919-0affea614482 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:21.102212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb14424-2ab5-4d3d-b919-0affea614482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:21.102534 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb14424-2ab5-4d3d-b919-0affea614482 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:21.103170 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb14424-2ab5-4d3d-b919-0affea614482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:21.104828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb14424-2ab5-4d3d-b919-0affea614482 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:21.115158 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1328/2650] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:35:21.119275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa16e40-42d2-4dd6-ac9b-30ef6a6ea0b0 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:21.129517 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa16e40-42d2-4dd6-ac9b-30ef6a6ea0b0 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:21.134204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ad2d0-64fe-4062-aba1-f7851272662b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:21.134842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ad2d0-64fe-4062-aba1-f7851272662b admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:21.135124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ad2d0-64fe-4062-aba1-f7851272662b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:21.135402 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ad2d0-64fe-4062-aba1-f7851272662b admin admin] PATH_INFO: `/v3/projects/022dbbd1e74d4c53b9f4b67f3702036a/users/cd0aa5a1035e4554949453046461c59f/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:21.138388 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:21.150625 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ad2d0-64fe-4062-aba1-f7851272662b admin admin] RBAC: Authorizing `identity:create_grant(project_id=022dbbd1e74d4c53b9f4b67f3702036a, user_id=cd0aa5a1035e4554949453046461c59f, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:21.153337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ad2d0-64fe-4062-aba1-f7851272662b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:21.247759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1323/2651] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:35:21 2022] PUT /identity/v3/projects/022dbbd1e74d4c53b9f4b67f3702036a/users/cd0aa5a1035e4554949453046461c59f/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:21.259739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b123c-c8a9-4821-a608-6d68443b3d3a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:21.260031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b123c-c8a9-4821-a608-6d68443b3d3a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:21.260348 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b123c-c8a9-4821-a608-6d68443b3d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:21.289483 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-fb1b123c-c8a9-4821-a608-6d68443b3d3a None None] MFA Rules not processed for user `cd0aa5a1035e4554949453046461c59f`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:21.291890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-fb1b123c-c8a9-4821-a608-6d68443b3d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:21.468872 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:21.477856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1329/2652] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:21.490158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ad2d0-64fe-4062-aba1-f7851272662b admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:21.491833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-618ad2d0-64fe-4062-aba1-f7851272662b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:21.550271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ad2d0-64fe-4062-aba1-f7851272662b admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:21.553867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b307f-5f76-41c5-b05e-5ff0f741eba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:21.554465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b307f-5f76-41c5-b05e-5ff0f741eba4 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:21.554746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b307f-5f76-41c5-b05e-5ff0f741eba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:21.555021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b307f-5f76-41c5-b05e-5ff0f741eba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:21.563188 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b307f-5f76-41c5-b05e-5ff0f741eba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:21.565186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b307f-5f76-41c5-b05e-5ff0f741eba4 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:21.573984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1324/2653] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:21.592860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b123c-c8a9-4821-a608-6d68443b3d3a None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:21.657154 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b123c-c8a9-4821-a608-6d68443b3d3a None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:21.661656 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:21.662270 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:21.662546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:21.662816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] PATH_INFO: `/v3/users/cd0aa5a1035e4554949453046461c59f` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:21.664013 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd0aa5a1035e4554949453046461c59f)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:21.665942 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:21.884091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd0aa5a1035e4554949453046461c59f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:21.890983 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd0aa5a1035e4554949453046461c59f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:21.979570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd0aa5a1035e4554949453046461c59f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:21.980236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd0aa5a1035e4554949453046461c59f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:21.992337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd0aa5a1035e4554949453046461c59f'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:21.998116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1330/2654] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:35:21 2022] DELETE /identity/v3/users/cd0aa5a1035e4554949453046461c59f => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:22.002607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b577d-fc87-4d36-945e-ed64a74dd627 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:22.003145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b577d-fc87-4d36-945e-ed64a74dd627 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:22.003145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b577d-fc87-4d36-945e-ed64a74dd627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:22.142513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-563b577d-fc87-4d36-945e-ed64a74dd627 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:22.143644 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-563b577d-fc87-4d36-945e-ed64a74dd627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:22.202147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1325/2655] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:22.212608 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:22.214006 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:22.291991 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f6abe-dd4a-47c9-8e42-61ebd59ea760 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:22.295520 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4659fb5-ea70-4170-a804-0344046cee90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:22.296108 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4659fb5-ea70-4170-a804-0344046cee90 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:22.296414 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4659fb5-ea70-4170-a804-0344046cee90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:22.296683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4659fb5-ea70-4170-a804-0344046cee90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:22.305631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4659fb5-ea70-4170-a804-0344046cee90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:22.307029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4659fb5-ea70-4170-a804-0344046cee90 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:22.310888 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:22.316771 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1331/2656] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:22.832783 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b577d-fc87-4d36-945e-ed64a74dd627 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:22.834299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-563b577d-fc87-4d36-945e-ed64a74dd627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:22.850638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b577d-fc87-4d36-945e-ed64a74dd627 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:22.860234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:22.860234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:22.860234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:22.860234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] PATH_INFO: `/v3/projects/022dbbd1e74d4c53b9f4b67f3702036a` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:22.860984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] RBAC: Authorizing `identity:delete_project(project_id=022dbbd1e74d4c53b9f4b67f3702036a)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:22.860984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:23.009166 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '022dbbd1e74d4c53b9f4b67f3702036a', 'request_id': 'req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:23.014717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '022dbbd1e74d4c53b9f4b67f3702036a', 'request_id': 'req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:23.105233 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '022dbbd1e74d4c53b9f4b67f3702036a', 'request_id': 'req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:23.117215 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] The token cache is being invalidate because project 022dbbd1e74d4c53b9f4b67f3702036a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:35:23.117506 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:23.118143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:23.119619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1326/2657] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:35:22 2022] DELETE /identity/v3/projects/022dbbd1e74d4c53b9f4b67f3702036a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:32.848080 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:33.114120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:33.736473 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4659fb5-ea70-4170-a804-0344046cee90 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:33.737949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-f4659fb5-ea70-4170-a804-0344046cee90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:33.837391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4659fb5-ea70-4170-a804-0344046cee90 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:33.898890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e281ae-3055-4652-8a67-845e179245d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:33.899480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e281ae-3055-4652-8a67-845e179245d2 service placement] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:33.899788 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e281ae-3055-4652-8a67-845e179245d2 service placement] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:33.900064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e281ae-3055-4652-8a67-845e179245d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:33.901893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b3e281ae-3055-4652-8a67-845e179245d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:33.914501 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e281ae-3055-4652-8a67-845e179245d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:33.916116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e281ae-3055-4652-8a67-845e179245d2 service placement] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:33.927576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1332/2658] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:38.850855 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:38.852287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:38.909271 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ccaa6a-57e9-4a35-89cd-7717858e7ddc admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:38.912801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300d2ea-85e8-423d-b1f3-a3e288be1d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:38.913389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300d2ea-85e8-423d-b1f3-a3e288be1d9d service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:38.913665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300d2ea-85e8-423d-b1f3-a3e288be1d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:38.913935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300d2ea-85e8-423d-b1f3-a3e288be1d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:38.915634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0300d2ea-85e8-423d-b1f3-a3e288be1d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:38.976660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300d2ea-85e8-423d-b1f3-a3e288be1d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:38.979369 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300d2ea-85e8-423d-b1f3-a3e288be1d9d service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:38.990430 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1327/2659] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:42.729011 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e281ae-3055-4652-8a67-845e179245d2 service placement] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:42.730453 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b3e281ae-3055-4652-8a67-845e179245d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:42.755873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e281ae-3055-4652-8a67-845e179245d2 service placement] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:42.755873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e72e5c-0e42-43b8-91f7-7dd50b508f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:42.755873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e72e5c-0e42-43b8-91f7-7dd50b508f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:42.755873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e72e5c-0e42-43b8-91f7-7dd50b508f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:42.756287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e72e5c-0e42-43b8-91f7-7dd50b508f76 admin admin] PATH_INFO: `/v3/users/859d36aa86bd403ca1d46b245efaef61` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:42.756287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e72e5c-0e42-43b8-91f7-7dd50b508f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=859d36aa86bd403ca1d46b245efaef61)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:42.756287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e72e5c-0e42-43b8-91f7-7dd50b508f76 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:42.982326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e0e72e5c-0e42-43b8-91f7-7dd50b508f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '859d36aa86bd403ca1d46b245efaef61'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:42.989292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e0e72e5c-0e42-43b8-91f7-7dd50b508f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '859d36aa86bd403ca1d46b245efaef61'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:43.070830 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e0e72e5c-0e42-43b8-91f7-7dd50b508f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '859d36aa86bd403ca1d46b245efaef61'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:43.071502 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e0e72e5c-0e42-43b8-91f7-7dd50b508f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '859d36aa86bd403ca1d46b245efaef61'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:43.076656 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:43.084791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e0e72e5c-0e42-43b8-91f7-7dd50b508f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '859d36aa86bd403ca1d46b245efaef61'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:43.090884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1333/2660] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:35:42 2022] DELETE /identity/v3/users/859d36aa86bd403ca1d46b245efaef61 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:43.358077 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300d2ea-85e8-423d-b1f3-a3e288be1d9d service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:43.368643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300d2ea-85e8-423d-b1f3-a3e288be1d9d service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:43.431928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea38537-9cb0-442e-8328-aba7f778008a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:43.432588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea38537-9cb0-442e-8328-aba7f778008a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:43.432897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea38537-9cb0-442e-8328-aba7f778008a admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:43.433174 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea38537-9cb0-442e-8328-aba7f778008a admin admin] PATH_INFO: `/v3/projects/e6f51ffa08494524b47ac220f1bd74de` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:43.434276 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea38537-9cb0-442e-8328-aba7f778008a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6f51ffa08494524b47ac220f1bd74de)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:43.436245 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea38537-9cb0-442e-8328-aba7f778008a admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:43.591099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-aea38537-9cb0-442e-8328-aba7f778008a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6f51ffa08494524b47ac220f1bd74de', 'request_id': 'req-aea38537-9cb0-442e-8328-aba7f778008a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:43.593007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:43.597765 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-aea38537-9cb0-442e-8328-aba7f778008a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6f51ffa08494524b47ac220f1bd74de', 'request_id': 'req-aea38537-9cb0-442e-8328-aba7f778008a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:43.682161 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-aea38537-9cb0-442e-8328-aba7f778008a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6f51ffa08494524b47ac220f1bd74de', 'request_id': 'req-aea38537-9cb0-442e-8328-aba7f778008a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:43.693817 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-aea38537-9cb0-442e-8328-aba7f778008a admin admin] The token cache is being invalidate because project e6f51ffa08494524b47ac220f1bd74de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:35:43.694113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-aea38537-9cb0-442e-8328-aba7f778008a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:43.694735 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-aea38537-9cb0-442e-8328-aba7f778008a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:43.696212 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1328/2661] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:35:43 2022] DELETE /identity/v3/projects/e6f51ffa08494524b47ac220f1bd74de => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:43.701216 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e14a3-88fa-44fe-abc8-93fa18c4ea83 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:43.701523 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e14a3-88fa-44fe-abc8-93fa18c4ea83 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:43.701791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e14a3-88fa-44fe-abc8-93fa18c4ea83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:43.722099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-f07e14a3-88fa-44fe-abc8-93fa18c4ea83 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:43.723308 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-f07e14a3-88fa-44fe-abc8-93fa18c4ea83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:43.782251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1334/2662] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:43.790437 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c70a40-f7b2-477f-8204-51b60f0e7c3d None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:43.791818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c70a40-f7b2-477f-8204-51b60f0e7c3d None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:43.791818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c70a40-f7b2-477f-8204-51b60f0e7c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:43.811121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-e0c70a40-f7b2-477f-8204-51b60f0e7c3d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:43.812381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-e0c70a40-f7b2-477f-8204-51b60f0e7c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:43.820726 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1329/2663] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:43.824961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e14a3-88fa-44fe-abc8-93fa18c4ea83 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:43.834628 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e14a3-88fa-44fe-abc8-93fa18c4ea83 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:43.839279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ec340-efe9-41e0-9097-f95df2bdd5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:43.839866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ec340-efe9-41e0-9097-f95df2bdd5cf admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:43.840148 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ec340-efe9-41e0-9097-f95df2bdd5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:43.840457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ec340-efe9-41e0-9097-f95df2bdd5cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:43.841097 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ec340-efe9-41e0-9097-f95df2bdd5cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:35:43.841429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ec340-efe9-41e0-9097-f95df2bdd5cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:43.843023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ec340-efe9-41e0-9097-f95df2bdd5cf admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:43.854468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1335/2664] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:35:43.869142 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c70a40-f7b2-477f-8204-51b60f0e7c3d None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:43.878875 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c70a40-f7b2-477f-8204-51b60f0e7c3d None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:43.883469 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f664bf3-c470-4bc0-82b2-1d5b1a7708ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:43.884050 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f664bf3-c470-4bc0-82b2-1d5b1a7708ab admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:43.884355 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f664bf3-c470-4bc0-82b2-1d5b1a7708ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:43.884628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f664bf3-c470-4bc0-82b2-1d5b1a7708ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:43.885278 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f664bf3-c470-4bc0-82b2-1d5b1a7708ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:43.887186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f664bf3-c470-4bc0-82b2-1d5b1a7708ab admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:43.985793 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1330/2665] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:35:43 2022] POST /identity/v3/projects => generated 355 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:43.990026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ec340-efe9-41e0-9097-f95df2bdd5cf admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:43.999863 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ec340-efe9-41e0-9097-f95df2bdd5cf admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:44.056304 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8213a-fc82-457e-917c-a76b63febd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:44.056920 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8213a-fc82-457e-917c-a76b63febd94 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:44.057203 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8213a-fc82-457e-917c-a76b63febd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:44.057480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8213a-fc82-457e-917c-a76b63febd94 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:44.058212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8213a-fc82-457e-917c-a76b63febd94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:44.060129 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8213a-fc82-457e-917c-a76b63febd94 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:44.207525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1336/2666] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:35:43 2022] POST /identity/v3/users => generated 278 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:44.211156 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f664bf3-c470-4bc0-82b2-1d5b1a7708ab admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:44.221319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f664bf3-c470-4bc0-82b2-1d5b1a7708ab admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:44.225829 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175012f9-a39e-4bfa-90c8-f30bb4b2ef30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:44.226422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175012f9-a39e-4bfa-90c8-f30bb4b2ef30 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:44.226703 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175012f9-a39e-4bfa-90c8-f30bb4b2ef30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:44.226976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175012f9-a39e-4bfa-90c8-f30bb4b2ef30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:44.227615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175012f9-a39e-4bfa-90c8-f30bb4b2ef30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:44.229217 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175012f9-a39e-4bfa-90c8-f30bb4b2ef30 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:44.239108 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1331/2667] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:44.242900 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8213a-fc82-457e-917c-a76b63febd94 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:44.253157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8213a-fc82-457e-917c-a76b63febd94 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:44.257766 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4065be6-b072-45d6-821b-102804a55c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:44.258348 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4065be6-b072-45d6-821b-102804a55c0f admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:44.258627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4065be6-b072-45d6-821b-102804a55c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:44.258899 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4065be6-b072-45d6-821b-102804a55c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:44.259531 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4065be6-b072-45d6-821b-102804a55c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:44.261138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4065be6-b072-45d6-821b-102804a55c0f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:44.272526 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1337/2668] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:44.276704 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175012f9-a39e-4bfa-90c8-f30bb4b2ef30 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:44.286760 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175012f9-a39e-4bfa-90c8-f30bb4b2ef30 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:44.291128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65657c4b-69d4-473c-9f53-6f11052ddcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:44.291747 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65657c4b-69d4-473c-9f53-6f11052ddcb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:44.292023 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65657c4b-69d4-473c-9f53-6f11052ddcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:44.292351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65657c4b-69d4-473c-9f53-6f11052ddcb7 admin admin] PATH_INFO: `/v3/projects/7da4ba21f240496e8f3a4454003800b7/users/0794535f92de47abb6126f340fe10e5c/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:44.306766 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65657c4b-69d4-473c-9f53-6f11052ddcb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7da4ba21f240496e8f3a4454003800b7, user_id=0794535f92de47abb6126f340fe10e5c, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:44.309415 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65657c4b-69d4-473c-9f53-6f11052ddcb7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:44.414477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1332/2669] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:35:44 2022] PUT /identity/v3/projects/7da4ba21f240496e8f3a4454003800b7/users/0794535f92de47abb6126f340fe10e5c/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:44.426792 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac96e5-46f7-4a49-a51d-4a2c1c9dbe12 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:44.427263 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac96e5-46f7-4a49-a51d-4a2c1c9dbe12 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:44.427660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac96e5-46f7-4a49-a51d-4a2c1c9dbe12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:44.455505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-c9ac96e5-46f7-4a49-a51d-4a2c1c9dbe12 None None] MFA Rules not processed for user `0794535f92de47abb6126f340fe10e5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:44.456742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c9ac96e5-46f7-4a49-a51d-4a2c1c9dbe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:44.495004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:44.517956 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1338/2670] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:44.522479 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65657c4b-69d4-473c-9f53-6f11052ddcb7 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:44.585705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65657c4b-69d4-473c-9f53-6f11052ddcb7 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:44.590446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b09ce5-92eb-468d-92ee-5a461813d5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:44.591056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b09ce5-92eb-468d-92ee-5a461813d5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:44.591337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b09ce5-92eb-468d-92ee-5a461813d5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:44.591610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b09ce5-92eb-468d-92ee-5a461813d5a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:44.592295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b09ce5-92eb-468d-92ee-5a461813d5a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:44.594873 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b09ce5-92eb-468d-92ee-5a461813d5a8 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:44.705800 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1333/2671] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:35:44 2022] POST /identity/v3/projects => generated 355 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:44.709785 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac96e5-46f7-4a49-a51d-4a2c1c9dbe12 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:44.721135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac96e5-46f7-4a49-a51d-4a2c1c9dbe12 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:44.776965 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb34d4-718f-4ebb-887c-bba3a69174bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:44.777468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb34d4-718f-4ebb-887c-bba3a69174bf admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:44.777749 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb34d4-718f-4ebb-887c-bba3a69174bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:44.778021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb34d4-718f-4ebb-887c-bba3a69174bf admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:44.778653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb34d4-718f-4ebb-887c-bba3a69174bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:44.780618 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb34d4-718f-4ebb-887c-bba3a69174bf admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:44.893177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1339/2672] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:35:44 2022] POST /identity/v3/users => generated 284 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:44.897357 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b09ce5-92eb-468d-92ee-5a461813d5a8 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:44.907873 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b09ce5-92eb-468d-92ee-5a461813d5a8 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:44.912433 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7a907-714e-4519-a80c-8b7383cef203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:44.913025 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7a907-714e-4519-a80c-8b7383cef203 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:44.913306 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7a907-714e-4519-a80c-8b7383cef203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:44.913580 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7a907-714e-4519-a80c-8b7383cef203 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:44.914221 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7a907-714e-4519-a80c-8b7383cef203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:44.915834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7a907-714e-4519-a80c-8b7383cef203 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:44.926580 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1334/2673] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:44.930708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb34d4-718f-4ebb-887c-bba3a69174bf admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:44.941204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb34d4-718f-4ebb-887c-bba3a69174bf admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:44.945694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce472d41-b798-42c0-a5f7-a9756d34b1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:44.946344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce472d41-b798-42c0-a5f7-a9756d34b1d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:44.946624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce472d41-b798-42c0-a5f7-a9756d34b1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:44.946893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce472d41-b798-42c0-a5f7-a9756d34b1d9 admin admin] PATH_INFO: `/v3/projects/c914f428266d43a494223d21f1b62b52/users/5422a4667d5d4f95b815010ead3effd3/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:44.962184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce472d41-b798-42c0-a5f7-a9756d34b1d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c914f428266d43a494223d21f1b62b52, user_id=5422a4667d5d4f95b815010ead3effd3, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:44.964872 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce472d41-b798-42c0-a5f7-a9756d34b1d9 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:45.048152 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1340/2674] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:35:44 2022] PUT /identity/v3/projects/c914f428266d43a494223d21f1b62b52/users/5422a4667d5d4f95b815010ead3effd3/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:45.052195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7a907-714e-4519-a80c-8b7383cef203 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:45.062178 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7a907-714e-4519-a80c-8b7383cef203 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:45.107400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:45.117084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee369b0c-6d53-4130-aa41-d003399fafa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:45.117677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369b0c-6d53-4130-aa41-d003399fafa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:45.118893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369b0c-6d53-4130-aa41-d003399fafa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:45.118893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369b0c-6d53-4130-aa41-d003399fafa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:45.119099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee369b0c-6d53-4130-aa41-d003399fafa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:45.120679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee369b0c-6d53-4130-aa41-d003399fafa0 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:45.130626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1335/2675] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:45.134977 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce472d41-b798-42c0-a5f7-a9756d34b1d9 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:45.145828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce472d41-b798-42c0-a5f7-a9756d34b1d9 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:45.150755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775ffd6-26ca-45b0-9d88-a3cd07b8f93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:45.151393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775ffd6-26ca-45b0-9d88-a3cd07b8f93e admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:45.151677 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775ffd6-26ca-45b0-9d88-a3cd07b8f93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:45.151952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775ffd6-26ca-45b0-9d88-a3cd07b8f93e admin admin] PATH_INFO: `/v3/projects/c914f428266d43a494223d21f1b62b52/users/5422a4667d5d4f95b815010ead3effd3/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:45.154057 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775ffd6-26ca-45b0-9d88-a3cd07b8f93e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c914f428266d43a494223d21f1b62b52, user_id=5422a4667d5d4f95b815010ead3effd3, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:45.159001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775ffd6-26ca-45b0-9d88-a3cd07b8f93e admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:45.167561 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1341/2676] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:35:45 2022] PUT /identity/v3/projects/c914f428266d43a494223d21f1b62b52/users/5422a4667d5d4f95b815010ead3effd3/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:45.178327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44652597-baf0-41f0-b87e-71a56eca065a None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:45.178838 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44652597-baf0-41f0-b87e-71a56eca065a None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:45.179295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44652597-baf0-41f0-b87e-71a56eca065a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:45.207325 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-44652597-baf0-41f0-b87e-71a56eca065a None None] MFA Rules not processed for user `5422a4667d5d4f95b815010ead3effd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:45.208552 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-44652597-baf0-41f0-b87e-71a56eca065a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:45.275561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1336/2677] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:35:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:45.287481 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775ffd6-26ca-45b0-9d88-a3cd07b8f93e admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:45.289189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9775ffd6-26ca-45b0-9d88-a3cd07b8f93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:45.365772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775ffd6-26ca-45b0-9d88-a3cd07b8f93e admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:45.369401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8185637-c6f6-4448-b1d2-cd318530f180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:45.370000 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8185637-c6f6-4448-b1d2-cd318530f180 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:45.370279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8185637-c6f6-4448-b1d2-cd318530f180 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:45.370551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8185637-c6f6-4448-b1d2-cd318530f180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:45.379307 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8185637-c6f6-4448-b1d2-cd318530f180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:45.380762 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8185637-c6f6-4448-b1d2-cd318530f180 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:45.390024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1342/2678] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:45.662170 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44652597-baf0-41f0-b87e-71a56eca065a None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:45.672032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44652597-baf0-41f0-b87e-71a56eca065a None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:45.675695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608aec43-8b43-4ac2-ae4d-028517204bc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:45.679929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608aec43-8b43-4ac2-ae4d-028517204bc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:45.679929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608aec43-8b43-4ac2-ae4d-028517204bc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:45.679929 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608aec43-8b43-4ac2-ae4d-028517204bc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:45.733385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608aec43-8b43-4ac2-ae4d-028517204bc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:45.735397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608aec43-8b43-4ac2-ae4d-028517204bc7 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:45.745370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1337/2679] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:55.385803 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=88,keystone:INSERT=5 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:55.741327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=68 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:02.934681 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491fed3-c833-4040-b1c1-76cb86bcf3b4 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:02.935210 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491fed3-c833-4040-b1c1-76cb86bcf3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:02.935210 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491fed3-c833-4040-b1c1-76cb86bcf3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:02.954326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3491fed3-c833-4040-b1c1-76cb86bcf3b4 None None] MFA Rules not processed for user `75a2f770cc774d61baa6c07532665477`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:02.956192 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3491fed3-c833-4040-b1c1-76cb86bcf3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:03.015026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1343/2680] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:36:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:04.983786 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608aec43-8b43-4ac2-ae4d-028517204bc7 service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:05.047998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608aec43-8b43-4ac2-ae4d-028517204bc7 service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:05.053544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:05.054178 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:05.054457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:05.054733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] PATH_INFO: `/v3/users/0794535f92de47abb6126f340fe10e5c` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:05.055858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0794535f92de47abb6126f340fe10e5c)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:05.057792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:05.329029 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0794535f92de47abb6126f340fe10e5c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:05.329683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0794535f92de47abb6126f340fe10e5c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:05.336700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0794535f92de47abb6126f340fe10e5c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:05.407687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0794535f92de47abb6126f340fe10e5c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:05.408925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0794535f92de47abb6126f340fe10e5c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:05.427505 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1338/2681] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:36:04 2022] DELETE /identity/v3/users/0794535f92de47abb6126f340fe10e5c => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:36:05.432072 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64166167-007a-4d0b-ad37-398273742395 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:05.432625 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64166167-007a-4d0b-ad37-398273742395 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:05.432726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64166167-007a-4d0b-ad37-398273742395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:05.451328 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-64166167-007a-4d0b-ad37-398273742395 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:05.452556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-64166167-007a-4d0b-ad37-398273742395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:05.509732 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1344/2682] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:05.520120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:05.521594 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:05.578746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3ed53-20b1-43f1-ac0a-469ae311e6f7 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:05.582905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1295b2-683f-4d2c-8c86-e36764ef5f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:05.583448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1295b2-683f-4d2c-8c86-e36764ef5f49 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:05.583728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1295b2-683f-4d2c-8c86-e36764ef5f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:05.584001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1295b2-683f-4d2c-8c86-e36764ef5f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:05.592778 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1295b2-683f-4d2c-8c86-e36764ef5f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:05.594212 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1295b2-683f-4d2c-8c86-e36764ef5f49 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:05.604458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1339/2683] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:36:06.100475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64166167-007a-4d0b-ad37-398273742395 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:06.101941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-64166167-007a-4d0b-ad37-398273742395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:06.118640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64166167-007a-4d0b-ad37-398273742395 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:06.123738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:06.124064 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:06.124389 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:06.124709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] PATH_INFO: `/v3/projects/7da4ba21f240496e8f3a4454003800b7` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:06.125857 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7da4ba21f240496e8f3a4454003800b7)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:06.128260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:06.282986 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7da4ba21f240496e8f3a4454003800b7', 'request_id': 'req-c4dfd657-d632-459d-840a-efc3acedf81a'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:06.289670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7da4ba21f240496e8f3a4454003800b7', 'request_id': 'req-c4dfd657-d632-459d-840a-efc3acedf81a'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:06.297367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7da4ba21f240496e8f3a4454003800b7', 'request_id': 'req-c4dfd657-d632-459d-840a-efc3acedf81a'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:06.424123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] The token cache is being invalidate because project 7da4ba21f240496e8f3a4454003800b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:36:06.432091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:06.432091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:06.432091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1345/2684] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:36:06 2022] DELETE /identity/v3/projects/7da4ba21f240496e8f3a4454003800b7 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:36:06.432890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1295b2-683f-4d2c-8c86-e36764ef5f49 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:06.440176 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1295b2-683f-4d2c-8c86-e36764ef5f49 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:06.463187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:06.505184 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb1e90-08bf-4184-8d7f-c9f3fa1f161d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:06.505855 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb1e90-08bf-4184-8d7f-c9f3fa1f161d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:06.506141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb1e90-08bf-4184-8d7f-c9f3fa1f161d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:06.506420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb1e90-08bf-4184-8d7f-c9f3fa1f161d admin admin] PATH_INFO: `/v3/users/5422a4667d5d4f95b815010ead3effd3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:06.507606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb1e90-08bf-4184-8d7f-c9f3fa1f161d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5422a4667d5d4f95b815010ead3effd3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:06.509534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb1e90-08bf-4184-8d7f-c9f3fa1f161d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:06.770182 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-28cb1e90-08bf-4184-8d7f-c9f3fa1f161d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5422a4667d5d4f95b815010ead3effd3'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:06.770842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-28cb1e90-08bf-4184-8d7f-c9f3fa1f161d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5422a4667d5d4f95b815010ead3effd3'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:06.777868 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-28cb1e90-08bf-4184-8d7f-c9f3fa1f161d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5422a4667d5d4f95b815010ead3effd3'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:06.865318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-28cb1e90-08bf-4184-8d7f-c9f3fa1f161d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5422a4667d5d4f95b815010ead3effd3'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:06.866042 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-28cb1e90-08bf-4184-8d7f-c9f3fa1f161d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5422a4667d5d4f95b815010ead3effd3'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:06.890305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1340/2685] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:36:06 2022] DELETE /identity/v3/users/5422a4667d5d4f95b815010ead3effd3 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:36:07.432350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:07.433839 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:07.450144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfd657-d632-459d-840a-efc3acedf81a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:07.480106 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:07.512566 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaef5f-9e7a-447a-9f20-1c4441925943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:07.513221 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaef5f-9e7a-447a-9f20-1c4441925943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:07.513508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaef5f-9e7a-447a-9f20-1c4441925943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:07.513784 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaef5f-9e7a-447a-9f20-1c4441925943 admin admin] PATH_INFO: `/v3/projects/c914f428266d43a494223d21f1b62b52` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:07.515048 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddaef5f-9e7a-447a-9f20-1c4441925943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c914f428266d43a494223d21f1b62b52)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:07.517034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddaef5f-9e7a-447a-9f20-1c4441925943 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:07.637967 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dddaef5f-9e7a-447a-9f20-1c4441925943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c914f428266d43a494223d21f1b62b52', 'request_id': 'req-dddaef5f-9e7a-447a-9f20-1c4441925943'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:07.644709 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dddaef5f-9e7a-447a-9f20-1c4441925943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c914f428266d43a494223d21f1b62b52', 'request_id': 'req-dddaef5f-9e7a-447a-9f20-1c4441925943'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:07.651045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dddaef5f-9e7a-447a-9f20-1c4441925943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c914f428266d43a494223d21f1b62b52', 'request_id': 'req-dddaef5f-9e7a-447a-9f20-1c4441925943'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:07.761342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dddaef5f-9e7a-447a-9f20-1c4441925943 admin admin] The token cache is being invalidate because project c914f428266d43a494223d21f1b62b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:36:07.761634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dddaef5f-9e7a-447a-9f20-1c4441925943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:07.762283 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dddaef5f-9e7a-447a-9f20-1c4441925943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:07.763760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1346/2686] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:36:07 2022] DELETE /identity/v3/projects/c914f428266d43a494223d21f1b62b52 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:36:07.768678 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a570-477b-46a0-a33e-ba2e7d877df3 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:07.769147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a570-477b-46a0-a33e-ba2e7d877df3 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:07.769336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a570-477b-46a0-a33e-ba2e7d877df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:07.788473 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-fd46a570-477b-46a0-a33e-ba2e7d877df3 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:07.789679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-fd46a570-477b-46a0-a33e-ba2e7d877df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:07.849031 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1341/2687] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:07.858267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac81fa1-0163-4a9b-8e13-5eeb529b9e65 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:07.858556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac81fa1-0163-4a9b-8e13-5eeb529b9e65 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:07.858831 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac81fa1-0163-4a9b-8e13-5eeb529b9e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:07.881338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-2ac81fa1-0163-4a9b-8e13-5eeb529b9e65 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:07.882660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-2ac81fa1-0163-4a9b-8e13-5eeb529b9e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:07.890855 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1347/2688] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:07.897619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46a570-477b-46a0-a33e-ba2e7d877df3 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:07.905701 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46a570-477b-46a0-a33e-ba2e7d877df3 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:07.911315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779326b9-c462-4bed-befd-55eb18b25ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:07.911561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779326b9-c462-4bed-befd-55eb18b25ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:07.911837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779326b9-c462-4bed-befd-55eb18b25ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:07.912107 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779326b9-c462-4bed-befd-55eb18b25ca6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:07.912780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779326b9-c462-4bed-befd-55eb18b25ca6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:36:07.913468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779326b9-c462-4bed-befd-55eb18b25ca6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:07.914739 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779326b9-c462-4bed-befd-55eb18b25ca6 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:07.927870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1342/2689] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:36:07.935232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac81fa1-0163-4a9b-8e13-5eeb529b9e65 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:07.946921 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac81fa1-0163-4a9b-8e13-5eeb529b9e65 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:07.952212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835257c3-6e66-4930-b95e-c3885c289427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:07.952871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835257c3-6e66-4930-b95e-c3885c289427 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:07.953149 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835257c3-6e66-4930-b95e-c3885c289427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:07.953419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835257c3-6e66-4930-b95e-c3885c289427 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:07.954087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835257c3-6e66-4930-b95e-c3885c289427 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:07.956058 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835257c3-6e66-4930-b95e-c3885c289427 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:08.086477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1348/2690] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:36:07 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:36:08.087945 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779326b9-c462-4bed-befd-55eb18b25ca6 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:08.098475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779326b9-c462-4bed-befd-55eb18b25ca6 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:08.128778 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:08.159697 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5cc9f-f0a1-4dd4-b41f-135331d027df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:08.160338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5cc9f-f0a1-4dd4-b41f-135331d027df admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:08.160626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5cc9f-f0a1-4dd4-b41f-135331d027df admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:08.160893 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5cc9f-f0a1-4dd4-b41f-135331d027df admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:08.161523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5cc9f-f0a1-4dd4-b41f-135331d027df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:08.163508 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5cc9f-f0a1-4dd4-b41f-135331d027df admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:08.356540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1343/2691] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:36:08 2022] POST /identity/v3/users => generated 290 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:36:08.360871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835257c3-6e66-4930-b95e-c3885c289427 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:08.372541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835257c3-6e66-4930-b95e-c3885c289427 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:08.378216 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7df09c-bd32-4380-a848-b8264bdf377a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:08.379135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7df09c-bd32-4380-a848-b8264bdf377a admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:08.379605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7df09c-bd32-4380-a848-b8264bdf377a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:08.380091 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7df09c-bd32-4380-a848-b8264bdf377a admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:08.380974 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7df09c-bd32-4380-a848-b8264bdf377a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:08.382896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7df09c-bd32-4380-a848-b8264bdf377a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:08.395311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1349/2692] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:36:08.399251 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5cc9f-f0a1-4dd4-b41f-135331d027df admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:08.410596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5cc9f-f0a1-4dd4-b41f-135331d027df admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:08.415652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba80a39-b1fd-473b-bc3f-569ea6264ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:08.416394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba80a39-b1fd-473b-bc3f-569ea6264ace admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:08.416745 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba80a39-b1fd-473b-bc3f-569ea6264ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:08.417092 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba80a39-b1fd-473b-bc3f-569ea6264ace admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:08.417809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba80a39-b1fd-473b-bc3f-569ea6264ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:08.419572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba80a39-b1fd-473b-bc3f-569ea6264ace admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:08.434559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1344/2693] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:36:08.436026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7df09c-bd32-4380-a848-b8264bdf377a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:08.448117 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7df09c-bd32-4380-a848-b8264bdf377a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:08.453396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cce33c-dcab-4d7d-8760-ee67847defcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:08.454063 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cce33c-dcab-4d7d-8760-ee67847defcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:08.454345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cce33c-dcab-4d7d-8760-ee67847defcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:08.454619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cce33c-dcab-4d7d-8760-ee67847defcf admin admin] PATH_INFO: `/v3/projects/5f3c46add2034ae38381f11ece410e62/users/a4f2d1a0b105407f8f4bebd7f777b7c5/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:08.471730 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cce33c-dcab-4d7d-8760-ee67847defcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3c46add2034ae38381f11ece410e62, user_id=a4f2d1a0b105407f8f4bebd7f777b7c5, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:08.474433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cce33c-dcab-4d7d-8760-ee67847defcf admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:08.533656 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1350/2694] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:36:08 2022] PUT /identity/v3/projects/5f3c46add2034ae38381f11ece410e62/users/a4f2d1a0b105407f8f4bebd7f777b7c5/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:36:08.544328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4447539-f796-4a6c-aec3-f4c5f147ac06 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:08.544917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4447539-f796-4a6c-aec3-f4c5f147ac06 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:08.545380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4447539-f796-4a6c-aec3-f4c5f147ac06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:08.576935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-c4447539-f796-4a6c-aec3-f4c5f147ac06 None None] MFA Rules not processed for user `a4f2d1a0b105407f8f4bebd7f777b7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:08.578293 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c4447539-f796-4a6c-aec3-f4c5f147ac06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:08.656442 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1345/2695] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:36:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:08.662100 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cce33c-dcab-4d7d-8760-ee67847defcf admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:08.731577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:08.737542 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cce33c-dcab-4d7d-8760-ee67847defcf admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:08.742670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31232c-07f9-4965-8198-c73c1486b8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:08.743302 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31232c-07f9-4965-8198-c73c1486b8cd admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:08.743582 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31232c-07f9-4965-8198-c73c1486b8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:08.746504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31232c-07f9-4965-8198-c73c1486b8cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:08.746504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31232c-07f9-4965-8198-c73c1486b8cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:08.746504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31232c-07f9-4965-8198-c73c1486b8cd admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:08.872251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1351/2696] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:36:08 2022] POST /identity/v3/projects => generated 381 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:36:08.876547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4447539-f796-4a6c-aec3-f4c5f147ac06 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:08.887652 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4447539-f796-4a6c-aec3-f4c5f147ac06 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:08.945598 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10464c-48b2-4742-afc9-a7c30b07476b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:08.946317 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10464c-48b2-4742-afc9-a7c30b07476b admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:08.946630 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10464c-48b2-4742-afc9-a7c30b07476b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:08.946918 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10464c-48b2-4742-afc9-a7c30b07476b admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:08.947548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10464c-48b2-4742-afc9-a7c30b07476b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:08.949548 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10464c-48b2-4742-afc9-a7c30b07476b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:09.054451 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1346/2697] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:36:08 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:36:09.058345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31232c-07f9-4965-8198-c73c1486b8cd admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:09.068891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31232c-07f9-4965-8198-c73c1486b8cd admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:09.073510 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec0272-de3d-4499-8fe7-bdd71d91276f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:09.074105 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec0272-de3d-4499-8fe7-bdd71d91276f admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:09.074386 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec0272-de3d-4499-8fe7-bdd71d91276f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:09.074699 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec0272-de3d-4499-8fe7-bdd71d91276f admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:09.075297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec0272-de3d-4499-8fe7-bdd71d91276f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:09.076939 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec0272-de3d-4499-8fe7-bdd71d91276f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:09.086694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1352/2698] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:36:09.090922 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10464c-48b2-4742-afc9-a7c30b07476b admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:09.101381 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10464c-48b2-4742-afc9-a7c30b07476b admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:09.106046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438eb36a-57c2-414d-bc94-75d85039055c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:09.106628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438eb36a-57c2-414d-bc94-75d85039055c admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:09.106909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438eb36a-57c2-414d-bc94-75d85039055c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:09.107185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438eb36a-57c2-414d-bc94-75d85039055c admin admin] PATH_INFO: `/v3/projects/75e4763988ea450f9fe301431da8d0cd/users/7608fd82b3b846a1abba1e14ab485774/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:09.122159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438eb36a-57c2-414d-bc94-75d85039055c admin admin] RBAC: Authorizing `identity:create_grant(project_id=75e4763988ea450f9fe301431da8d0cd, user_id=7608fd82b3b846a1abba1e14ab485774, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:09.124851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438eb36a-57c2-414d-bc94-75d85039055c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:09.196426 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1347/2699] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:36:09 2022] PUT /identity/v3/projects/75e4763988ea450f9fe301431da8d0cd/users/7608fd82b3b846a1abba1e14ab485774/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:36:09.200415 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec0272-de3d-4499-8fe7-bdd71d91276f admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:09.211358 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec0272-de3d-4499-8fe7-bdd71d91276f admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:09.266184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435dbf1-7b4d-4051-b30e-ac4fd6524ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:09.266789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435dbf1-7b4d-4051-b30e-ac4fd6524ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:09.267067 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435dbf1-7b4d-4051-b30e-ac4fd6524ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:09.267334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435dbf1-7b4d-4051-b30e-ac4fd6524ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:09.268527 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435dbf1-7b4d-4051-b30e-ac4fd6524ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:09.270123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435dbf1-7b4d-4051-b30e-ac4fd6524ab0 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:09.280005 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1353/2700] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:36:09.284684 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438eb36a-57c2-414d-bc94-75d85039055c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:09.289975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:09.295123 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438eb36a-57c2-414d-bc94-75d85039055c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:09.299722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be1be7-a758-4c9b-b640-a3e645ec4407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:09.300465 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be1be7-a758-4c9b-b640-a3e645ec4407 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:09.300725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be1be7-a758-4c9b-b640-a3e645ec4407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:09.300994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be1be7-a758-4c9b-b640-a3e645ec4407 admin admin] PATH_INFO: `/v3/projects/75e4763988ea450f9fe301431da8d0cd/users/7608fd82b3b846a1abba1e14ab485774/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:09.309545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be1be7-a758-4c9b-b640-a3e645ec4407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75e4763988ea450f9fe301431da8d0cd, user_id=7608fd82b3b846a1abba1e14ab485774, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:09.309545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be1be7-a758-4c9b-b640-a3e645ec4407 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:09.315422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1348/2701] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:36:09 2022] PUT /identity/v3/projects/75e4763988ea450f9fe301431da8d0cd/users/7608fd82b3b846a1abba1e14ab485774/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:36:09.326428 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9825d7e-7ee1-4356-b287-ca1ba2be3b82 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:09.326717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9825d7e-7ee1-4356-b287-ca1ba2be3b82 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:09.326998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9825d7e-7ee1-4356-b287-ca1ba2be3b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:09.355394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-c9825d7e-7ee1-4356-b287-ca1ba2be3b82 None None] MFA Rules not processed for user `7608fd82b3b846a1abba1e14ab485774`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:09.356697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c9825d7e-7ee1-4356-b287-ca1ba2be3b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:09.423695 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1354/2702] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:36:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:09.435886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be1be7-a758-4c9b-b640-a3e645ec4407 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:09.437727 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-38be1be7-a758-4c9b-b640-a3e645ec4407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:09.495998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be1be7-a758-4c9b-b640-a3e645ec4407 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:09.499761 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db339c-e90a-4e43-ac39-19a33bf16316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:09.501147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db339c-e90a-4e43-ac39-19a33bf16316 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:09.501434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db339c-e90a-4e43-ac39-19a33bf16316 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:09.501733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db339c-e90a-4e43-ac39-19a33bf16316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:09.511408 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db339c-e90a-4e43-ac39-19a33bf16316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:09.512931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db339c-e90a-4e43-ac39-19a33bf16316 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:09.524667 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1349/2703] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:36:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:36:10.119248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9825d7e-7ee1-4356-b287-ca1ba2be3b82 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:10.170062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:10.183897 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9825d7e-7ee1-4356-b287-ca1ba2be3b82 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:10.188537 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:10.189138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:10.189419 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:10.189720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] PATH_INFO: `/v3/users/a4f2d1a0b105407f8f4bebd7f777b7c5` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:10.190851 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4f2d1a0b105407f8f4bebd7f777b7c5)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:10.192789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:10.466545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4f2d1a0b105407f8f4bebd7f777b7c5'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:10.473379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4f2d1a0b105407f8f4bebd7f777b7c5'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:10.549968 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4f2d1a0b105407f8f4bebd7f777b7c5'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:10.550577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4f2d1a0b105407f8f4bebd7f777b7c5'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:10.562688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4f2d1a0b105407f8f4bebd7f777b7c5'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:10.569196 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1355/2704] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:36:10 2022] DELETE /identity/v3/users/a4f2d1a0b105407f8f4bebd7f777b7c5 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:36:10.573604 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8619f-49cc-4795-88c6-dfcdb52321ec None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:10.574183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8619f-49cc-4795-88c6-dfcdb52321ec None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:10.574183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8619f-49cc-4795-88c6-dfcdb52321ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:10.593321 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-1ed8619f-49cc-4795-88c6-dfcdb52321ec None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:10.594540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1ed8619f-49cc-4795-88c6-dfcdb52321ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:10.652530 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1350/2705] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:10.662505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:10.663898 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:10.720253 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80060ea-818b-4cfa-8dd0-36c846bc14ae admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:10.723796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774fb19-3173-4c55-9948-7f080197cd5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:10.724423 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774fb19-3173-4c55-9948-7f080197cd5a service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:10.724707 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774fb19-3173-4c55-9948-7f080197cd5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:10.724982 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774fb19-3173-4c55-9948-7f080197cd5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:10.733864 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774fb19-3173-4c55-9948-7f080197cd5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:10.735292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774fb19-3173-4c55-9948-7f080197cd5a service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:10.745540 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1356/2706] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:36:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:36:11.209195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8619f-49cc-4795-88c6-dfcdb52321ec None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:11.210661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1ed8619f-49cc-4795-88c6-dfcdb52321ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:11.226501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8619f-49cc-4795-88c6-dfcdb52321ec None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:11.231580 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:11.232370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:11.232650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:11.232991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] PATH_INFO: `/v3/projects/5f3c46add2034ae38381f11ece410e62` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:11.234121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f3c46add2034ae38381f11ece410e62)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:11.236132 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:11.347556 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f3c46add2034ae38381f11ece410e62', 'request_id': 'req-8604b978-1963-414a-b601-b24847383d83'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:11.353114 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f3c46add2034ae38381f11ece410e62', 'request_id': 'req-8604b978-1963-414a-b601-b24847383d83'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:11.442300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f3c46add2034ae38381f11ece410e62', 'request_id': 'req-8604b978-1963-414a-b601-b24847383d83'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:11.454331 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] The token cache is being invalidate because project 5f3c46add2034ae38381f11ece410e62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:36:11.454658 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:11.455805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:11.457318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1351/2707] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:36:11 2022] DELETE /identity/v3/projects/5f3c46add2034ae38381f11ece410e62 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:36:11.461360 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774fb19-3173-4c55-9948-7f080197cd5a service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:11.472083 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774fb19-3173-4c55-9948-7f080197cd5a service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:11.532300 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c958327-4e3a-4291-969a-111327dea5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:11.532933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c958327-4e3a-4291-969a-111327dea5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:11.533208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c958327-4e3a-4291-969a-111327dea5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:11.534186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c958327-4e3a-4291-969a-111327dea5cf admin admin] PATH_INFO: `/v3/users/7608fd82b3b846a1abba1e14ab485774` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:11.535504 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c958327-4e3a-4291-969a-111327dea5cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7608fd82b3b846a1abba1e14ab485774)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:11.537372 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c958327-4e3a-4291-969a-111327dea5cf admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:11.553401 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:11.730926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8c958327-4e3a-4291-969a-111327dea5cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7608fd82b3b846a1abba1e14ab485774'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:11.738057 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8c958327-4e3a-4291-969a-111327dea5cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7608fd82b3b846a1abba1e14ab485774'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:11.888311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8c958327-4e3a-4291-969a-111327dea5cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7608fd82b3b846a1abba1e14ab485774'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:11.888922 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8c958327-4e3a-4291-969a-111327dea5cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7608fd82b3b846a1abba1e14ab485774'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:11.902153 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-8c958327-4e3a-4291-969a-111327dea5cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7608fd82b3b846a1abba1e14ab485774'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:11.908181 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1357/2708] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:36:11 2022] DELETE /identity/v3/users/7608fd82b3b846a1abba1e14ab485774 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:36:12.415432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:12.416889 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:12.430025 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604b978-1963-414a-b601-b24847383d83 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:12.442137 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:12.488073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:12.488737 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:12.489017 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:12.489356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce admin admin] PATH_INFO: `/v3/projects/75e4763988ea450f9fe301431da8d0cd` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:12.490458 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=75e4763988ea450f9fe301431da8d0cd)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:12.492466 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:12.607499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75e4763988ea450f9fe301431da8d0cd', 'request_id': 'req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:12.613205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75e4763988ea450f9fe301431da8d0cd', 'request_id': 'req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:12.687761 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75e4763988ea450f9fe301431da8d0cd', 'request_id': 'req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:12.705890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce admin admin] The token cache is being invalidate because project 75e4763988ea450f9fe301431da8d0cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:36:12.706178 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:12.707374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3b9fdfbd-2edd-4113-8e46-4098bdba67ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:12.708928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1352/2709] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:36:12 2022] DELETE /identity/v3/projects/75e4763988ea450f9fe301431da8d0cd => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:36:12.713941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48df821-f2df-4aff-8c6f-10242ab577d0 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:12.714440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48df821-f2df-4aff-8c6f-10242ab577d0 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:12.714620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48df821-f2df-4aff-8c6f-10242ab577d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:12.735090 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-a48df821-f2df-4aff-8c6f-10242ab577d0 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:12.736335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a48df821-f2df-4aff-8c6f-10242ab577d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:12.795927 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1358/2710] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:12.803878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8837d2-0fc9-472b-9f0c-73a85dadb2d5 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:12.804183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8837d2-0fc9-472b-9f0c-73a85dadb2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:12.804480 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8837d2-0fc9-472b-9f0c-73a85dadb2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:12.824082 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-3b8837d2-0fc9-472b-9f0c-73a85dadb2d5 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:12.825287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3b8837d2-0fc9-472b-9f0c-73a85dadb2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:12.833702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1353/2711] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:12.837756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48df821-f2df-4aff-8c6f-10242ab577d0 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:12.847184 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48df821-f2df-4aff-8c6f-10242ab577d0 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:12.851737 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e42b8-58f1-4161-a713-79edc43054d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:12.852492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e42b8-58f1-4161-a713-79edc43054d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:12.852775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e42b8-58f1-4161-a713-79edc43054d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:12.853048 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e42b8-58f1-4161-a713-79edc43054d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:12.853692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e42b8-58f1-4161-a713-79edc43054d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:36:12.853993 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e42b8-58f1-4161-a713-79edc43054d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:12.855647 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e42b8-58f1-4161-a713-79edc43054d5 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:12.867130 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1359/2712] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:36:12.871380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8837d2-0fc9-472b-9f0c-73a85dadb2d5 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:12.881045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8837d2-0fc9-472b-9f0c-73a85dadb2d5 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:12.885640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0e152-dde7-40cc-8dd1-9e36bf08d056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:12.886235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0e152-dde7-40cc-8dd1-9e36bf08d056 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:12.886508 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0e152-dde7-40cc-8dd1-9e36bf08d056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:12.886775 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0e152-dde7-40cc-8dd1-9e36bf08d056 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:12.887400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0e152-dde7-40cc-8dd1-9e36bf08d056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:12.889341 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0e152-dde7-40cc-8dd1-9e36bf08d056 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:13.028808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1354/2713] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:36:12 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:36:13.033347 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e42b8-58f1-4161-a713-79edc43054d5 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:13.046530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e42b8-58f1-4161-a713-79edc43054d5 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:13.112436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51909c2c-37be-47da-bc86-954676feb55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:13.113165 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51909c2c-37be-47da-bc86-954676feb55f admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:13.113317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51909c2c-37be-47da-bc86-954676feb55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:13.113584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51909c2c-37be-47da-bc86-954676feb55f admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:13.114220 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51909c2c-37be-47da-bc86-954676feb55f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:13.116555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51909c2c-37be-47da-bc86-954676feb55f admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:13.214584 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1360/2714] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:36:13 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:36:13.218661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0e152-dde7-40cc-8dd1-9e36bf08d056 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:13.230344 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0e152-dde7-40cc-8dd1-9e36bf08d056 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:13.235224 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f95ddf-3cef-474f-9cf8-b6ad91daf8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:13.236056 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f95ddf-3cef-474f-9cf8-b6ad91daf8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:13.236366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f95ddf-3cef-474f-9cf8-b6ad91daf8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:13.236640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f95ddf-3cef-474f-9cf8-b6ad91daf8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:13.237281 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f95ddf-3cef-474f-9cf8-b6ad91daf8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:13.239314 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f95ddf-3cef-474f-9cf8-b6ad91daf8a6 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:13.250931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1355/2715] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:36:13.261121 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51909c2c-37be-47da-bc86-954676feb55f admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:13.271693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51909c2c-37be-47da-bc86-954676feb55f admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:13.277004 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1818f-cdab-4853-982b-21777154cb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:13.277640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1818f-cdab-4853-982b-21777154cb73 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:13.277923 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1818f-cdab-4853-982b-21777154cb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:13.278201 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1818f-cdab-4853-982b-21777154cb73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:13.278847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1818f-cdab-4853-982b-21777154cb73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:13.280590 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1818f-cdab-4853-982b-21777154cb73 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:13.282894 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:13.292874 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1361/2716] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:36:13.302089 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f95ddf-3cef-474f-9cf8-b6ad91daf8a6 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:13.312518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f95ddf-3cef-474f-9cf8-b6ad91daf8a6 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:13.317690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ce973-08b4-48cd-a9fb-d8acfb853862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:13.318315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ce973-08b4-48cd-a9fb-d8acfb853862 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:13.318589 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ce973-08b4-48cd-a9fb-d8acfb853862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:13.318860 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ce973-08b4-48cd-a9fb-d8acfb853862 admin admin] PATH_INFO: `/v3/projects/a697de0f7a4d4b718d46d97f17c813b7/users/299a032386cb43c381f3f418c142d30f/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:13.334752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ce973-08b4-48cd-a9fb-d8acfb853862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a697de0f7a4d4b718d46d97f17c813b7, user_id=299a032386cb43c381f3f418c142d30f, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:13.350088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ce973-08b4-48cd-a9fb-d8acfb853862 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:13.482870 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1356/2717] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:36:13 2022] PUT /identity/v3/projects/a697de0f7a4d4b718d46d97f17c813b7/users/299a032386cb43c381f3f418c142d30f/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:36:13.488223 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d4686-3abc-4130-8d36-b72482158f02 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:13.488543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d4686-3abc-4130-8d36-b72482158f02 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:13.488822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d4686-3abc-4130-8d36-b72482158f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:13.509835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-c79d4686-3abc-4130-8d36-b72482158f02 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:13.510973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c79d4686-3abc-4130-8d36-b72482158f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:13.570693 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1362/2718] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:13.581094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ce973-08b4-48cd-a9fb-d8acfb853862 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:13.582860 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-922ce973-08b4-48cd-a9fb-d8acfb853862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:13.631503 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:13.643671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ce973-08b4-48cd-a9fb-d8acfb853862 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:13.647454 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e7a80-30f1-4b76-bec0-7d0a88db9613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:13.648354 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e7a80-30f1-4b76-bec0-7d0a88db9613 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:13.648638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e7a80-30f1-4b76-bec0-7d0a88db9613 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:13.648909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e7a80-30f1-4b76-bec0-7d0a88db9613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:13.657689 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e7a80-30f1-4b76-bec0-7d0a88db9613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:13.659148 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e7a80-30f1-4b76-bec0-7d0a88db9613 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:13.668501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1357/2719] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:36:15.128553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f8c54-9833-4e73-9a41-14fad21620bd None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:15.129066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f8c54-9833-4e73-9a41-14fad21620bd None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:15.129460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f8c54-9833-4e73-9a41-14fad21620bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:15.159331 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-910f8c54-9833-4e73-9a41-14fad21620bd None None] MFA Rules not processed for user `299a032386cb43c381f3f418c142d30f`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:15.160645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-910f8c54-9833-4e73-9a41-14fad21620bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:15.232108 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1363/2720] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:36:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:15.243370 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e7a80-30f1-4b76-bec0-7d0a88db9613 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:15.244793 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c69e7a80-30f1-4b76-bec0-7d0a88db9613 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:15.302080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e7a80-30f1-4b76-bec0-7d0a88db9613 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:15.305645 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d92a9-ff9c-4de7-ab34-941d44039cd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:15.306242 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d92a9-ff9c-4de7-ab34-941d44039cd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:15.306518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d92a9-ff9c-4de7-ab34-941d44039cd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:15.306787 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d92a9-ff9c-4de7-ab34-941d44039cd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:15.314950 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d92a9-ff9c-4de7-ab34-941d44039cd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:15.316393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d92a9-ff9c-4de7-ab34-941d44039cd5 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:15.325748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1358/2721] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:36:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:36:17.181342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93052d-19d2-4b0c-93c2-46e20c2245b7 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:17.181844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93052d-19d2-4b0c-93c2-46e20c2245b7 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:17.182125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93052d-19d2-4b0c-93c2-46e20c2245b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:17.203538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-1d93052d-19d2-4b0c-93c2-46e20c2245b7 None None] MFA Rules not processed for user `299a032386cb43c381f3f418c142d30f`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:17.204924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-1d93052d-19d2-4b0c-93c2-46e20c2245b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:17.214831 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1364/2722] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:36:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:25.321124 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=33 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:27.196558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=63 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:34.442712 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4da35-5cef-44bd-afbb-d2d1a6945a66 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:34.443216 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4da35-5cef-44bd-afbb-d2d1a6945a66 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:34.443330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4da35-5cef-44bd-afbb-d2d1a6945a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:34.462690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-c3a4da35-5cef-44bd-afbb-d2d1a6945a66 None None] MFA Rules not processed for user `299a032386cb43c381f3f418c142d30f`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:34.463925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c3a4da35-5cef-44bd-afbb-d2d1a6945a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:34.470746 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1359/2723] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:36:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:44.456515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:57.456897 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93052d-19d2-4b0c-93c2-46e20c2245b7 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:57.458311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-1d93052d-19d2-4b0c-93c2-46e20c2245b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:57.470206 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93052d-19d2-4b0c-93c2-46e20c2245b7 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:57.474439 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fb091-7b45-4aca-81c9-c2b4c3cdf31f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:57.475028 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fb091-7b45-4aca-81c9-c2b4c3cdf31f service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:57.475305 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fb091-7b45-4aca-81c9-c2b4c3cdf31f service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:57.475579 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fb091-7b45-4aca-81c9-c2b4c3cdf31f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:57.477256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7d1fb091-7b45-4aca-81c9-c2b4c3cdf31f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:57.564596 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fb091-7b45-4aca-81c9-c2b4c3cdf31f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:57.566060 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fb091-7b45-4aca-81c9-c2b4c3cdf31f service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:57.586263 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1365/2724] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:36:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:36:58.439957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb923f6-4b5a-4fd6-84e2-43b4ddd95029 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:58.440513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb923f6-4b5a-4fd6-84e2-43b4ddd95029 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:58.440618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb923f6-4b5a-4fd6-84e2-43b4ddd95029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:58.470468 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-9cb923f6-4b5a-4fd6-84e2-43b4ddd95029 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:58.471592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9cb923f6-4b5a-4fd6-84e2-43b4ddd95029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:58.525330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1360/2725] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:36:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:58.685697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fb091-7b45-4aca-81c9-c2b4c3cdf31f service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:58.687114 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7d1fb091-7b45-4aca-81c9-c2b4c3cdf31f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:58.754225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fb091-7b45-4aca-81c9-c2b4c3cdf31f service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:58.759858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f579b7-8940-430f-88cc-0cf9a0854861 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:58.760493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f579b7-8940-430f-88cc-0cf9a0854861 service swift] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:58.760776 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f579b7-8940-430f-88cc-0cf9a0854861 service swift] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:58.761053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f579b7-8940-430f-88cc-0cf9a0854861 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:58.769030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f579b7-8940-430f-88cc-0cf9a0854861 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:58.770631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f579b7-8940-430f-88cc-0cf9a0854861 service swift] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:58.778951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1366/2726] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:36:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:37:05.852075 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb923f6-4b5a-4fd6-84e2-43b4ddd95029 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:05.862481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb923f6-4b5a-4fd6-84e2-43b4ddd95029 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:05.866918 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd89a36-650d-47e5-b1ea-4d681a188a38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:05.867585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd89a36-650d-47e5-b1ea-4d681a188a38 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:05.867879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd89a36-650d-47e5-b1ea-4d681a188a38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:05.868309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd89a36-650d-47e5-b1ea-4d681a188a38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:05.870204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-ccd89a36-650d-47e5-b1ea-4d681a188a38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:05.886707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd89a36-650d-47e5-b1ea-4d681a188a38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:05.889821 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd89a36-650d-47e5-b1ea-4d681a188a38 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:05.903943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1361/2727] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:37:08.775627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=78 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:15.895178 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=35 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:16.637550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb2c8-64eb-44f2-bf2a-a1fcea3599d5 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:16.638041 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb2c8-64eb-44f2-bf2a-a1fcea3599d5 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:16.638177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb2c8-64eb-44f2-bf2a-a1fcea3599d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:16.687849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-828eb2c8-64eb-44f2-bf2a-a1fcea3599d5 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:16.689101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-828eb2c8-64eb-44f2-bf2a-a1fcea3599d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:16.696918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1367/2728] 10.222.0.85 () {58 vars in 967 bytes} [Mon Aug 29 20:37:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:16.709022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd89a36-650d-47e5-b1ea-4d681a188a38 service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:16.720311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd89a36-650d-47e5-b1ea-4d681a188a38 service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:16.725226 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ce66c-1d95-4379-bd70-010e3ca44ae6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:16.725842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ce66c-1d95-4379-bd70-010e3ca44ae6 service swift] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:16.726121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ce66c-1d95-4379-bd70-010e3ca44ae6 service swift] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:16.726397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ce66c-1d95-4379-bd70-010e3ca44ae6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:16.734396 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ce66c-1d95-4379-bd70-010e3ca44ae6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:16.735909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ce66c-1d95-4379-bd70-010e3ca44ae6 service swift] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:16.744483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1362/2729] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:37:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:37:17.538724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f4234-7789-4d06-9c7e-baa27c0f210c None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:17.539005 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f4234-7789-4d06-9c7e-baa27c0f210c None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:17.539275 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f4234-7789-4d06-9c7e-baa27c0f210c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:17.558281 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-237f4234-7789-4d06-9c7e-baa27c0f210c None None] MFA Rules not processed for user `75a2f770cc774d61baa6c07532665477`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:17.559571 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-237f4234-7789-4d06-9c7e-baa27c0f210c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:17.614933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1368/2730] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:37:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:37:20.212351 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ce66c-1d95-4379-bd70-010e3ca44ae6 service swift] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:20.223171 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ce66c-1d95-4379-bd70-010e3ca44ae6 service swift] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:20.228493 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee524329-fbbf-493a-9ed1-0f1827aa4444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:20.228639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee524329-fbbf-493a-9ed1-0f1827aa4444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:20.228844 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee524329-fbbf-493a-9ed1-0f1827aa4444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:20.229120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee524329-fbbf-493a-9ed1-0f1827aa4444 admin admin] PATH_INFO: `/v3/users/299a032386cb43c381f3f418c142d30f` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:20.230391 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee524329-fbbf-493a-9ed1-0f1827aa4444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=299a032386cb43c381f3f418c142d30f)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:20.232336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee524329-fbbf-493a-9ed1-0f1827aa4444 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:20.413889 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ee524329-fbbf-493a-9ed1-0f1827aa4444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299a032386cb43c381f3f418c142d30f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:20.415274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ee524329-fbbf-493a-9ed1-0f1827aa4444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299a032386cb43c381f3f418c142d30f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:20.423153 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ee524329-fbbf-493a-9ed1-0f1827aa4444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299a032386cb43c381f3f418c142d30f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:20.526795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ee524329-fbbf-493a-9ed1-0f1827aa4444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299a032386cb43c381f3f418c142d30f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:20.527422 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-ee524329-fbbf-493a-9ed1-0f1827aa4444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299a032386cb43c381f3f418c142d30f'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:20.545915 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1363/2731] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:37:20 2022] DELETE /identity/v3/users/299a032386cb43c381f3f418c142d30f => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:37:20.819692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f4234-7789-4d06-9c7e-baa27c0f210c None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:20.833626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f4234-7789-4d06-9c7e-baa27c0f210c None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:20.900858 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a04ce-d31b-4495-b258-99e40cd81674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:20.901491 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a04ce-d31b-4495-b258-99e40cd81674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:20.901786 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a04ce-d31b-4495-b258-99e40cd81674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:20.902087 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a04ce-d31b-4495-b258-99e40cd81674 admin admin] PATH_INFO: `/v3/projects/a697de0f7a4d4b718d46d97f17c813b7` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:20.903287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a04ce-d31b-4495-b258-99e40cd81674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a697de0f7a4d4b718d46d97f17c813b7)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:20.905388 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a04ce-d31b-4495-b258-99e40cd81674 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:21.089490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b56a04ce-d31b-4495-b258-99e40cd81674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a697de0f7a4d4b718d46d97f17c813b7', 'request_id': 'req-b56a04ce-d31b-4495-b258-99e40cd81674'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:21.098871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b56a04ce-d31b-4495-b258-99e40cd81674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a697de0f7a4d4b718d46d97f17c813b7', 'request_id': 'req-b56a04ce-d31b-4495-b258-99e40cd81674'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:21.106361 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b56a04ce-d31b-4495-b258-99e40cd81674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a697de0f7a4d4b718d46d97f17c813b7', 'request_id': 'req-b56a04ce-d31b-4495-b258-99e40cd81674'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:21.178055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b56a04ce-d31b-4495-b258-99e40cd81674 admin admin] The token cache is being invalidate because project a697de0f7a4d4b718d46d97f17c813b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:37:21.178572 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b56a04ce-d31b-4495-b258-99e40cd81674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:21.179221 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-b56a04ce-d31b-4495-b258-99e40cd81674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:21.180905 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1369/2732] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:37:20 2022] DELETE /identity/v3/projects/a697de0f7a4d4b718d46d97f17c813b7 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:37:21.186513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2277f1-4724-4ae4-af18-a828a4bd7299 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:21.186801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2277f1-4724-4ae4-af18-a828a4bd7299 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:21.187101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2277f1-4724-4ae4-af18-a828a4bd7299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:21.207044 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-1f2277f1-4724-4ae4-af18-a828a4bd7299 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:21.212935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1f2277f1-4724-4ae4-af18-a828a4bd7299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:21.271409 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1364/2733] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:21.280475 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ac358-5088-4ddc-a8f1-22a07368ab02 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:21.280752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ac358-5088-4ddc-a8f1-22a07368ab02 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:21.280978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ac358-5088-4ddc-a8f1-22a07368ab02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:21.290636 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:21.301237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-021ac358-5088-4ddc-a8f1-22a07368ab02 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:21.302653 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-021ac358-5088-4ddc-a8f1-22a07368ab02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:21.311263 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1370/2734] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:21.315474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2277f1-4724-4ae4-af18-a828a4bd7299 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:21.325847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2277f1-4724-4ae4-af18-a828a4bd7299 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:21.330753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae66ddd-bda0-4e0e-9167-2d4fcbda44e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:21.334292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae66ddd-bda0-4e0e-9167-2d4fcbda44e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:21.334292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae66ddd-bda0-4e0e-9167-2d4fcbda44e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:21.334292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae66ddd-bda0-4e0e-9167-2d4fcbda44e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:21.334292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae66ddd-bda0-4e0e-9167-2d4fcbda44e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:37:21.334292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae66ddd-bda0-4e0e-9167-2d4fcbda44e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:21.334837 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae66ddd-bda0-4e0e-9167-2d4fcbda44e7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:21.346529 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1365/2735] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:37:21.351493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ac358-5088-4ddc-a8f1-22a07368ab02 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:21.361436 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ac358-5088-4ddc-a8f1-22a07368ab02 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:21.366744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa342d-11fd-42a5-98aa-3a07760fea7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:21.367333 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa342d-11fd-42a5-98aa-3a07760fea7c admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:21.367612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa342d-11fd-42a5-98aa-3a07760fea7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:21.367884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa342d-11fd-42a5-98aa-3a07760fea7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:21.368546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa342d-11fd-42a5-98aa-3a07760fea7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:21.370474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa342d-11fd-42a5-98aa-3a07760fea7c admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:21.541157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1371/2736] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:37:21 2022] POST /identity/v3/projects => generated 369 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:37:21.607000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae66ddd-bda0-4e0e-9167-2d4fcbda44e7 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:21.623143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae66ddd-bda0-4e0e-9167-2d4fcbda44e7 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:21.669771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:21.684237 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1815116-d010-4bb0-ad06-b31234777b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:21.685594 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1815116-d010-4bb0-ad06-b31234777b95 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:21.685594 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1815116-d010-4bb0-ad06-b31234777b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:21.685594 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1815116-d010-4bb0-ad06-b31234777b95 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:21.686047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1815116-d010-4bb0-ad06-b31234777b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:21.687992 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1815116-d010-4bb0-ad06-b31234777b95 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:21.790741 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1366/2737] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:37:21 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:37:21.790961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa342d-11fd-42a5-98aa-3a07760fea7c admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:21.800624 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa342d-11fd-42a5-98aa-3a07760fea7c admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:21.805498 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8628a-69f5-4e21-9bd4-90502f388c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:21.806230 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8628a-69f5-4e21-9bd4-90502f388c7b admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:21.806570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8628a-69f5-4e21-9bd4-90502f388c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:21.806906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8628a-69f5-4e21-9bd4-90502f388c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:21.807634 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8628a-69f5-4e21-9bd4-90502f388c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:21.809473 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8628a-69f5-4e21-9bd4-90502f388c7b admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:21.820319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1372/2738] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:37:21.824642 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1815116-d010-4bb0-ad06-b31234777b95 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:21.835916 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1815116-d010-4bb0-ad06-b31234777b95 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:21.840878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f3858-86e2-44fa-b6b9-ce2d8bf31ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:21.841486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f3858-86e2-44fa-b6b9-ce2d8bf31ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:21.841771 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f3858-86e2-44fa-b6b9-ce2d8bf31ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:21.842098 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f3858-86e2-44fa-b6b9-ce2d8bf31ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:21.842751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f3858-86e2-44fa-b6b9-ce2d8bf31ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:21.844659 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f3858-86e2-44fa-b6b9-ce2d8bf31ea7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:21.856310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1367/2739] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:37:21.863045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8628a-69f5-4e21-9bd4-90502f388c7b admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:21.874877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8628a-69f5-4e21-9bd4-90502f388c7b admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:21.879963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5e34c-371d-491b-9b38-be7c23168039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:21.880662 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5e34c-371d-491b-9b38-be7c23168039 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:21.880945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5e34c-371d-491b-9b38-be7c23168039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:21.881219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5e34c-371d-491b-9b38-be7c23168039 admin admin] PATH_INFO: `/v3/projects/3dabffb1aab9479b8217a1429d31580b/users/ae6bbccb20b54201a79a08b6af92571b/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:21.898589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5e34c-371d-491b-9b38-be7c23168039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dabffb1aab9479b8217a1429d31580b, user_id=ae6bbccb20b54201a79a08b6af92571b, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:21.901357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5e34c-371d-491b-9b38-be7c23168039 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:21.955193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1373/2740] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:37:21 2022] PUT /identity/v3/projects/3dabffb1aab9479b8217a1429d31580b/users/ae6bbccb20b54201a79a08b6af92571b/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:37:21.956393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f72f0-6b12-4193-8a2d-12457cc7d5f9 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:21.956687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f72f0-6b12-4193-8a2d-12457cc7d5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:21.956968 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f72f0-6b12-4193-8a2d-12457cc7d5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:21.986380 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-8d9f72f0-6b12-4193-8a2d-12457cc7d5f9 None None] MFA Rules not processed for user `ae6bbccb20b54201a79a08b6af92571b`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:21.987614 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8d9f72f0-6b12-4193-8a2d-12457cc7d5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:22.053854 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1368/2741] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:37:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:37:22.065282 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5e34c-371d-491b-9b38-be7c23168039 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:22.066958 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a8b5e34c-371d-491b-9b38-be7c23168039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:22.125589 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5e34c-371d-491b-9b38-be7c23168039 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:22.129343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7e0e3-0324-45d3-8ccb-35b6bdd22ab6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7e768fdd1054e40b6b4910e39a6e533', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:22.129945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7e0e3-0324-45d3-8ccb-35b6bdd22ab6 service glance] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:22.130225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7e0e3-0324-45d3-8ccb-35b6bdd22ab6 service glance] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:22.130497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7e0e3-0324-45d3-8ccb-35b6bdd22ab6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:22.139311 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7e0e3-0324-45d3-8ccb-35b6bdd22ab6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:22.140873 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7e0e3-0324-45d3-8ccb-35b6bdd22ab6 service glance] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:22.151901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1374/2742] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:37:31.596759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f72f0-6b12-4193-8a2d-12457cc7d5f9 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:31.598200 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-8d9f72f0-6b12-4193-8a2d-12457cc7d5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:31.694315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f72f0-6b12-4193-8a2d-12457cc7d5f9 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:31.699832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df821823-f608-48cb-836e-edbf973c4e00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:31.700489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821823-f608-48cb-836e-edbf973c4e00 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:31.700773 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821823-f608-48cb-836e-edbf973c4e00 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:31.701051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821823-f608-48cb-836e-edbf973c4e00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:31.702854 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-df821823-f608-48cb-836e-edbf973c4e00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:31.715752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df821823-f608-48cb-836e-edbf973c4e00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:31.717441 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df821823-f608-48cb-836e-edbf973c4e00 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:31.729639 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1369/2743] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:37:31.824019 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7e0e3-0324-45d3-8ccb-35b6bdd22ab6 service glance] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:31.825615 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-87c7e0e3-0324-45d3-8ccb-35b6bdd22ab6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:31.893281 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7e0e3-0324-45d3-8ccb-35b6bdd22ab6 service glance] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:31.896989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56917db2-b171-4b8d-9db2-02a727a85cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:31.897593 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56917db2-b171-4b8d-9db2-02a727a85cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:31.897879 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56917db2-b171-4b8d-9db2-02a727a85cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:31.898146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56917db2-b171-4b8d-9db2-02a727a85cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:31.900674 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-56917db2-b171-4b8d-9db2-02a727a85cf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:31.910760 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56917db2-b171-4b8d-9db2-02a727a85cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:31.912225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56917db2-b171-4b8d-9db2-02a727a85cf0 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:31.921118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1375/2744] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:37:41.725054 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:41.917478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=81,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:51.805253 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df821823-f608-48cb-836e-edbf973c4e00 service nova] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:51.866125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df821823-f608-48cb-836e-edbf973c4e00 service nova] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:51.870662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:51.871886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:51.872158 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:51.872450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] PATH_INFO: `/v3/users/ae6bbccb20b54201a79a08b6af92571b` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:51.873553 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae6bbccb20b54201a79a08b6af92571b)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:51.875438 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:52.093523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae6bbccb20b54201a79a08b6af92571b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:52.094096 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae6bbccb20b54201a79a08b6af92571b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:52.100830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae6bbccb20b54201a79a08b6af92571b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:52.171327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae6bbccb20b54201a79a08b6af92571b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:52.171959 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae6bbccb20b54201a79a08b6af92571b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:52.189541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1370/2745] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:37:51 2022] DELETE /identity/v3/users/ae6bbccb20b54201a79a08b6af92571b => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:37:52.194026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edf9a7-7434-48ad-b3a4-3e34a43c7717 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:52.194545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edf9a7-7434-48ad-b3a4-3e34a43c7717 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:52.194545 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edf9a7-7434-48ad-b3a4-3e34a43c7717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:52.212922 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-06edf9a7-7434-48ad-b3a4-3e34a43c7717 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:52.214107 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-06edf9a7-7434-48ad-b3a4-3e34a43c7717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:52.271264 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1376/2746] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:52.281697 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:52.283322 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:52.339611 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07057c2-74b8-4267-afd1-56da6ad54bb8 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:52.343799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5975a-6d93-4147-88ba-c89122d2931f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:52.344368 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5975a-6d93-4147-88ba-c89122d2931f service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:52.344653 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5975a-6d93-4147-88ba-c89122d2931f service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:52.344927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5975a-6d93-4147-88ba-c89122d2931f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:52.353785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5975a-6d93-4147-88ba-c89122d2931f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:52.355222 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5975a-6d93-4147-88ba-c89122d2931f service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:52.365263 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1371/2747] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:37:52.886344 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edf9a7-7434-48ad-b3a4-3e34a43c7717 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:52.887791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-06edf9a7-7434-48ad-b3a4-3e34a43c7717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:52.900178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edf9a7-7434-48ad-b3a4-3e34a43c7717 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:52.905376 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae80372-c510-4cdc-8cb6-226e53681f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:52.906024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80372-c510-4cdc-8cb6-226e53681f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:52.906292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80372-c510-4cdc-8cb6-226e53681f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:52.906565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80372-c510-4cdc-8cb6-226e53681f78 admin admin] PATH_INFO: `/v3/projects/3dabffb1aab9479b8217a1429d31580b` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:52.907688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae80372-c510-4cdc-8cb6-226e53681f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dabffb1aab9479b8217a1429d31580b)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:52.909637 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae80372-c510-4cdc-8cb6-226e53681f78 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:53.016893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dae80372-c510-4cdc-8cb6-226e53681f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dabffb1aab9479b8217a1429d31580b', 'request_id': 'req-dae80372-c510-4cdc-8cb6-226e53681f78'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:53.023166 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dae80372-c510-4cdc-8cb6-226e53681f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dabffb1aab9479b8217a1429d31580b', 'request_id': 'req-dae80372-c510-4cdc-8cb6-226e53681f78'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:53.029583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dae80372-c510-4cdc-8cb6-226e53681f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dabffb1aab9479b8217a1429d31580b', 'request_id': 'req-dae80372-c510-4cdc-8cb6-226e53681f78'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:53.114050 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dae80372-c510-4cdc-8cb6-226e53681f78 admin admin] The token cache is being invalidate because project 3dabffb1aab9479b8217a1429d31580b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:37:53.114343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dae80372-c510-4cdc-8cb6-226e53681f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:53.114969 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-dae80372-c510-4cdc-8cb6-226e53681f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:53.116583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1377/2748] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:37:52 2022] DELETE /identity/v3/projects/3dabffb1aab9479b8217a1429d31580b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:37:53.126283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ed18e-9517-44a4-9a77-0bd584f31652 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:53.126283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ed18e-9517-44a4-9a77-0bd584f31652 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:53.126283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ed18e-9517-44a4-9a77-0bd584f31652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:53.142063 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f18ed18e-9517-44a4-9a77-0bd584f31652 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:53.143198 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f18ed18e-9517-44a4-9a77-0bd584f31652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:53.160392 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:53.200899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1372/2749] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:37:53.208295 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981e6bc-a74f-4b79-967e-28f0e1907d93 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:53.208796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981e6bc-a74f-4b79-967e-28f0e1907d93 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:53.208938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981e6bc-a74f-4b79-967e-28f0e1907d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:53.229236 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-7981e6bc-a74f-4b79-967e-28f0e1907d93 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:53.230424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7981e6bc-a74f-4b79-967e-28f0e1907d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:53.238742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1378/2750] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:53.242784 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ed18e-9517-44a4-9a77-0bd584f31652 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:53.252492 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ed18e-9517-44a4-9a77-0bd584f31652 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:53.257000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb602b-19cf-4ab9-b894-3b74b92fcfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:53.257587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb602b-19cf-4ab9-b894-3b74b92fcfc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:53.257885 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb602b-19cf-4ab9-b894-3b74b92fcfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:53.258157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb602b-19cf-4ab9-b894-3b74b92fcfc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:53.258794 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb602b-19cf-4ab9-b894-3b74b92fcfc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:37:53.259214 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb602b-19cf-4ab9-b894-3b74b92fcfc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:53.261002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb602b-19cf-4ab9-b894-3b74b92fcfc8 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:53.274026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1373/2751] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:37:53.278797 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981e6bc-a74f-4b79-967e-28f0e1907d93 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:53.287763 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981e6bc-a74f-4b79-967e-28f0e1907d93 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:53.292432 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f36a2-d88e-4ebd-9bfc-37852fbd0817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:53.292941 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f36a2-d88e-4ebd-9bfc-37852fbd0817 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:53.293218 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f36a2-d88e-4ebd-9bfc-37852fbd0817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:53.293489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f36a2-d88e-4ebd-9bfc-37852fbd0817 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:53.294111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f36a2-d88e-4ebd-9bfc-37852fbd0817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:53.295998 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f36a2-d88e-4ebd-9bfc-37852fbd0817 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:53.394353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1379/2752] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:37:53 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:37:53.398694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb602b-19cf-4ab9-b894-3b74b92fcfc8 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:53.409196 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb602b-19cf-4ab9-b894-3b74b92fcfc8 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:53.465890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6b006-7020-495c-8356-d3b2686ebcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:53.466487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6b006-7020-495c-8356-d3b2686ebcf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:53.466796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6b006-7020-495c-8356-d3b2686ebcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:53.467115 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6b006-7020-495c-8356-d3b2686ebcf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:53.467748 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6b006-7020-495c-8356-d3b2686ebcf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:53.469729 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6b006-7020-495c-8356-d3b2686ebcf2 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:53.571695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1374/2753] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:37:53 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:37:53.576494 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f36a2-d88e-4ebd-9bfc-37852fbd0817 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:53.588638 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f36a2-d88e-4ebd-9bfc-37852fbd0817 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:53.593232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefb699-2ef5-41af-b6b8-f54e4820ee65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:53.599178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefb699-2ef5-41af-b6b8-f54e4820ee65 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:53.599178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefb699-2ef5-41af-b6b8-f54e4820ee65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:53.599178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefb699-2ef5-41af-b6b8-f54e4820ee65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:53.599178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefb699-2ef5-41af-b6b8-f54e4820ee65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:53.599178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefb699-2ef5-41af-b6b8-f54e4820ee65 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:53.608563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1380/2754] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:37:53.612624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6b006-7020-495c-8356-d3b2686ebcf2 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:53.623304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6b006-7020-495c-8356-d3b2686ebcf2 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:53.628269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd000c-43b1-47fa-bd8e-2ddabccc9f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:53.628876 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd000c-43b1-47fa-bd8e-2ddabccc9f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:53.629159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd000c-43b1-47fa-bd8e-2ddabccc9f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:53.629436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd000c-43b1-47fa-bd8e-2ddabccc9f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:53.630078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd000c-43b1-47fa-bd8e-2ddabccc9f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:53.631671 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd000c-43b1-47fa-bd8e-2ddabccc9f12 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:53.643947 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1375/2755] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:37:53.648674 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefb699-2ef5-41af-b6b8-f54e4820ee65 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:53.658632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefb699-2ef5-41af-b6b8-f54e4820ee65 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:53.663334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015725c-c2fc-4f95-b0ee-4339d44abbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:53.663978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015725c-c2fc-4f95-b0ee-4339d44abbb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:53.664297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015725c-c2fc-4f95-b0ee-4339d44abbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:53.664576 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015725c-c2fc-4f95-b0ee-4339d44abbb1 admin admin] PATH_INFO: `/v3/projects/7122ce12a274434da9cfd0a4dd8c3d39/users/0ea4186145474e6c864cc405cbe64934/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:53.678965 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015725c-c2fc-4f95-b0ee-4339d44abbb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7122ce12a274434da9cfd0a4dd8c3d39, user_id=0ea4186145474e6c864cc405cbe64934, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:53.681635 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015725c-c2fc-4f95-b0ee-4339d44abbb1 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:53.789334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1381/2756] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:37:53 2022] PUT /identity/v3/projects/7122ce12a274434da9cfd0a4dd8c3d39/users/0ea4186145474e6c864cc405cbe64934/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:37:53.801149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95734cd8-6b01-47d3-a71f-20e9c3e0b7de None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:53.801434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95734cd8-6b01-47d3-a71f-20e9c3e0b7de None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:53.801708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95734cd8-6b01-47d3-a71f-20e9c3e0b7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:53.831035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-95734cd8-6b01-47d3-a71f-20e9c3e0b7de None None] MFA Rules not processed for user `0ea4186145474e6c864cc405cbe64934`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:53.832636 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-95734cd8-6b01-47d3-a71f-20e9c3e0b7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:53.894498 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1376/2757] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:37:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:53.906470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015725c-c2fc-4f95-b0ee-4339d44abbb1 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:53.908240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-b015725c-c2fc-4f95-b0ee-4339d44abbb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:53.929387 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:53.963742 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015725c-c2fc-4f95-b0ee-4339d44abbb1 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:53.968679 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9463196-2cb8-4090-a1e3-c5fc90eacfb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:53.969271 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9463196-2cb8-4090-a1e3-c5fc90eacfb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:53.969549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9463196-2cb8-4090-a1e3-c5fc90eacfb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:53.969822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9463196-2cb8-4090-a1e3-c5fc90eacfb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:53.977893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9463196-2cb8-4090-a1e3-c5fc90eacfb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:53.979317 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9463196-2cb8-4090-a1e3-c5fc90eacfb0 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:53.988024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1382/2758] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:37:54.012672 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95734cd8-6b01-47d3-a71f-20e9c3e0b7de None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:54.031963 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:54.075101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95734cd8-6b01-47d3-a71f-20e9c3e0b7de None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:54.079897 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:54.080540 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:54.080830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:54.081093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] PATH_INFO: `/v3/users/0ea4186145474e6c864cc405cbe64934` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:54.082206 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea4186145474e6c864cc405cbe64934)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:54.084256 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:54.276314 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea4186145474e6c864cc405cbe64934'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:54.276986 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea4186145474e6c864cc405cbe64934'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:54.284018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea4186145474e6c864cc405cbe64934'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:54.364497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea4186145474e6c864cc405cbe64934'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:54.365145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea4186145474e6c864cc405cbe64934'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:54.382670 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1377/2759] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:37:54 2022] DELETE /identity/v3/users/0ea4186145474e6c864cc405cbe64934 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:37:54.387123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579867d9-97be-40c8-a7a2-3fd39b4e8c21 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:54.387604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579867d9-97be-40c8-a7a2-3fd39b4e8c21 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:54.387714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579867d9-97be-40c8-a7a2-3fd39b4e8c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:54.406814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-579867d9-97be-40c8-a7a2-3fd39b4e8c21 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:54.408010 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-579867d9-97be-40c8-a7a2-3fd39b4e8c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:54.466318 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1383/2760] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:54.476055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:54.477484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:54.534178 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c78ec-5298-4681-9660-6063d77ecb90 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:54.537726 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340f3a3-c798-467b-a920-37680d1430cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:54.538321 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340f3a3-c798-467b-a920-37680d1430cd service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:54.538603 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340f3a3-c798-467b-a920-37680d1430cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:54.538879 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340f3a3-c798-467b-a920-37680d1430cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:54.548526 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340f3a3-c798-467b-a920-37680d1430cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:54.549954 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340f3a3-c798-467b-a920-37680d1430cd service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:54.559172 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1378/2761] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:37:54.988975 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579867d9-97be-40c8-a7a2-3fd39b4e8c21 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:54.990459 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-579867d9-97be-40c8-a7a2-3fd39b4e8c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:55.003890 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579867d9-97be-40c8-a7a2-3fd39b4e8c21 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:55.008789 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f1f4d-68d6-40e4-8abb-e27665587be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:55.009427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f1f4d-68d6-40e4-8abb-e27665587be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:55.009701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f1f4d-68d6-40e4-8abb-e27665587be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:55.009967 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f1f4d-68d6-40e4-8abb-e27665587be7 admin admin] PATH_INFO: `/v3/projects/7122ce12a274434da9cfd0a4dd8c3d39` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:55.011120 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f1f4d-68d6-40e4-8abb-e27665587be7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7122ce12a274434da9cfd0a4dd8c3d39)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:55.013804 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f1f4d-68d6-40e4-8abb-e27665587be7 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:55.118497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2f5f1f4d-68d6-40e4-8abb-e27665587be7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7122ce12a274434da9cfd0a4dd8c3d39', 'request_id': 'req-2f5f1f4d-68d6-40e4-8abb-e27665587be7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:55.125234 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2f5f1f4d-68d6-40e4-8abb-e27665587be7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7122ce12a274434da9cfd0a4dd8c3d39', 'request_id': 'req-2f5f1f4d-68d6-40e4-8abb-e27665587be7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:55.130627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2f5f1f4d-68d6-40e4-8abb-e27665587be7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7122ce12a274434da9cfd0a4dd8c3d39', 'request_id': 'req-2f5f1f4d-68d6-40e4-8abb-e27665587be7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:55.204115 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2f5f1f4d-68d6-40e4-8abb-e27665587be7 admin admin] The token cache is being invalidate because project 7122ce12a274434da9cfd0a4dd8c3d39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:37:55.204437 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2f5f1f4d-68d6-40e4-8abb-e27665587be7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:55.205092 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2f5f1f4d-68d6-40e4-8abb-e27665587be7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:37:55.206572 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1384/2762] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:37:54 2022] DELETE /identity/v3/projects/7122ce12a274434da9cfd0a4dd8c3d39 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:37:55.211781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101dbb72-e0ba-42a0-ba99-2b024080b80c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:55.213391 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101dbb72-e0ba-42a0-ba99-2b024080b80c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:55.213391 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101dbb72-e0ba-42a0-ba99-2b024080b80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:55.231501 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-101dbb72-e0ba-42a0-ba99-2b024080b80c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:55.232753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-101dbb72-e0ba-42a0-ba99-2b024080b80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:55.267881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:55.292575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1379/2763] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:55.300399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf025ea-160e-41a3-a3b8-cd31340296ea None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:55.300686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf025ea-160e-41a3-a3b8-cd31340296ea None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:55.300965 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf025ea-160e-41a3-a3b8-cd31340296ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:55.320817 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-baf025ea-160e-41a3-a3b8-cd31340296ea None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:55.321935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-baf025ea-160e-41a3-a3b8-cd31340296ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:55.330442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1385/2764] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:55.334508 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101dbb72-e0ba-42a0-ba99-2b024080b80c None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:55.343934 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101dbb72-e0ba-42a0-ba99-2b024080b80c None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:55.348509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ebf48-d9b3-4a22-8252-ab2109090027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:55.349096 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ebf48-d9b3-4a22-8252-ab2109090027 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:55.349378 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ebf48-d9b3-4a22-8252-ab2109090027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:55.349660 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ebf48-d9b3-4a22-8252-ab2109090027 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:55.350295 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ebf48-d9b3-4a22-8252-ab2109090027 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:37:55.350597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ebf48-d9b3-4a22-8252-ab2109090027 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:55.352192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ebf48-d9b3-4a22-8252-ab2109090027 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:55.364305 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1380/2765] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:37:55.368101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf025ea-160e-41a3-a3b8-cd31340296ea None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:55.377756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf025ea-160e-41a3-a3b8-cd31340296ea None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:55.382393 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858756f-dbb6-46e2-810c-72ffc291fa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:55.382978 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858756f-dbb6-46e2-810c-72ffc291fa69 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:55.383255 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858756f-dbb6-46e2-810c-72ffc291fa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:55.383526 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858756f-dbb6-46e2-810c-72ffc291fa69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:55.384142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858756f-dbb6-46e2-810c-72ffc291fa69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:55.386105 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858756f-dbb6-46e2-810c-72ffc291fa69 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:55.505095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1386/2766] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:37:55 2022] POST /identity/v3/projects => generated 391 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:37:55.509162 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ebf48-d9b3-4a22-8252-ab2109090027 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:55.519234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ebf48-d9b3-4a22-8252-ab2109090027 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:55.576820 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315bda8-b20f-4395-861c-3611500a42ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:55.576820 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315bda8-b20f-4395-861c-3611500a42ba admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:55.576820 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315bda8-b20f-4395-861c-3611500a42ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:55.576820 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315bda8-b20f-4395-861c-3611500a42ba admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:55.577358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315bda8-b20f-4395-861c-3611500a42ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:55.577908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315bda8-b20f-4395-861c-3611500a42ba admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:55.652122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1381/2767] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:37:55 2022] POST /identity/v3/users => generated 296 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:37:55.664062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858756f-dbb6-46e2-810c-72ffc291fa69 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:55.675020 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858756f-dbb6-46e2-810c-72ffc291fa69 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:55.679710 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a727c-dcd4-481a-8b46-bafd89c4cadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:55.680341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a727c-dcd4-481a-8b46-bafd89c4cadb admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:55.680620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a727c-dcd4-481a-8b46-bafd89c4cadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:55.680893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a727c-dcd4-481a-8b46-bafd89c4cadb admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:55.681535 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a727c-dcd4-481a-8b46-bafd89c4cadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:55.683116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a727c-dcd4-481a-8b46-bafd89c4cadb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:55.693583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1387/2768] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:37:55.697495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315bda8-b20f-4395-861c-3611500a42ba admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:55.707435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315bda8-b20f-4395-861c-3611500a42ba admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:55.711985 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66845760-7f58-44f6-9a72-37546ec8dd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:55.712597 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66845760-7f58-44f6-9a72-37546ec8dd3f admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:55.712899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66845760-7f58-44f6-9a72-37546ec8dd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:55.713167 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66845760-7f58-44f6-9a72-37546ec8dd3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:55.713808 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66845760-7f58-44f6-9a72-37546ec8dd3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:55.715878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66845760-7f58-44f6-9a72-37546ec8dd3f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:55.726586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1382/2769] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:37:55.731210 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a727c-dcd4-481a-8b46-bafd89c4cadb admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:55.740938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a727c-dcd4-481a-8b46-bafd89c4cadb admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:55.745500 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d68a4e-f36b-4905-b820-45f39dd4f063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:55.746126 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d68a4e-f36b-4905-b820-45f39dd4f063 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:55.746400 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d68a4e-f36b-4905-b820-45f39dd4f063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:55.746668 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d68a4e-f36b-4905-b820-45f39dd4f063 admin admin] PATH_INFO: `/v3/projects/4efdf978f25d43b3ae7e99d317fc3ec5/users/374561f619cb41fa9b188b61ab50f83a/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:55.748850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:55.761652 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d68a4e-f36b-4905-b820-45f39dd4f063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4efdf978f25d43b3ae7e99d317fc3ec5, user_id=374561f619cb41fa9b188b61ab50f83a, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:55.764339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d68a4e-f36b-4905-b820-45f39dd4f063 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:55.858793 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1388/2770] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:37:55 2022] PUT /identity/v3/projects/4efdf978f25d43b3ae7e99d317fc3ec5/users/374561f619cb41fa9b188b61ab50f83a/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:37:55.870267 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c15c2-9e7f-45f5-812c-60f48b91ac74 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:55.870556 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c15c2-9e7f-45f5-812c-60f48b91ac74 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:55.870833 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c15c2-9e7f-45f5-812c-60f48b91ac74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:55.898687 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-532c15c2-9e7f-45f5-812c-60f48b91ac74 None None] MFA Rules not processed for user `374561f619cb41fa9b188b61ab50f83a`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:55.899894 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-532c15c2-9e7f-45f5-812c-60f48b91ac74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:55.963268 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1383/2771] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:37:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:55.974468 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d68a4e-f36b-4905-b820-45f39dd4f063 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:55.976143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e3d68a4e-f36b-4905-b820-45f39dd4f063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:56.031282 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d68a4e-f36b-4905-b820-45f39dd4f063 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:56.034874 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a53ff-5fba-4b60-af0b-a6b14c5c2664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:56.035463 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a53ff-5fba-4b60-af0b-a6b14c5c2664 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:56.035756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a53ff-5fba-4b60-af0b-a6b14c5c2664 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:56.036013 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a53ff-5fba-4b60-af0b-a6b14c5c2664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:56.044837 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a53ff-5fba-4b60-af0b-a6b14c5c2664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:56.046279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a53ff-5fba-4b60-af0b-a6b14c5c2664 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:56.054754 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1389/2772] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:38:05.960379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:06.050886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:17.054691 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c15c2-9e7f-45f5-812c-60f48b91ac74 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:17.117146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c15c2-9e7f-45f5-812c-60f48b91ac74 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:17.121592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:17.122204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:17.122495 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:17.122752 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] PATH_INFO: `/v3/users/374561f619cb41fa9b188b61ab50f83a` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:17.123866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=374561f619cb41fa9b188b61ab50f83a)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:17.125829 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:17.430796 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '374561f619cb41fa9b188b61ab50f83a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:17.433266 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '374561f619cb41fa9b188b61ab50f83a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:17.438504 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '374561f619cb41fa9b188b61ab50f83a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:17.571166 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '374561f619cb41fa9b188b61ab50f83a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:17.571797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '374561f619cb41fa9b188b61ab50f83a'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:17.589679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1384/2773] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:38:17 2022] DELETE /identity/v3/users/374561f619cb41fa9b188b61ab50f83a => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:38:17.595843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52205a1f-ad3c-4987-80e3-14a1a9a5ed35 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:17.595843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52205a1f-ad3c-4987-80e3-14a1a9a5ed35 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:17.595843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52205a1f-ad3c-4987-80e3-14a1a9a5ed35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:17.615212 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-52205a1f-ad3c-4987-80e3-14a1a9a5ed35 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:38:17.616142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-52205a1f-ad3c-4987-80e3-14a1a9a5ed35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:17.674441 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1390/2774] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:38:17.684469 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:17.685838 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:17.745016 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a08e1-777f-4f2e-97ec-9d9e4c152685 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:17.748634 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b4d9d-abae-4025-ab83-7a41e02d90ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:17.749230 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b4d9d-abae-4025-ab83-7a41e02d90ae service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:17.749512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b4d9d-abae-4025-ab83-7a41e02d90ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:17.749786 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b4d9d-abae-4025-ab83-7a41e02d90ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:17.758559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b4d9d-abae-4025-ab83-7a41e02d90ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:17.759978 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b4d9d-abae-4025-ab83-7a41e02d90ae service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:17.769721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1385/2775] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:38:18.213298 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52205a1f-ad3c-4987-80e3-14a1a9a5ed35 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:18.214824 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-52205a1f-ad3c-4987-80e3-14a1a9a5ed35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:18.226413 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52205a1f-ad3c-4987-80e3-14a1a9a5ed35 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:18.233204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:18.233660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:18.233947 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:18.234342 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] PATH_INFO: `/v3/projects/4efdf978f25d43b3ae7e99d317fc3ec5` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:18.235585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4efdf978f25d43b3ae7e99d317fc3ec5)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:18.237549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:18.383551 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4efdf978f25d43b3ae7e99d317fc3ec5', 'request_id': 'req-d278de44-e43a-4fed-9e67-d62a024c0b8c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:18.389835 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4efdf978f25d43b3ae7e99d317fc3ec5', 'request_id': 'req-d278de44-e43a-4fed-9e67-d62a024c0b8c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:18.395316 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4efdf978f25d43b3ae7e99d317fc3ec5', 'request_id': 'req-d278de44-e43a-4fed-9e67-d62a024c0b8c'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:18.518811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] The token cache is being invalidate because project 4efdf978f25d43b3ae7e99d317fc3ec5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:38:18.519024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:18.519815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:18.521422 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1391/2776] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:38:18 2022] DELETE /identity/v3/projects/4efdf978f25d43b3ae7e99d317fc3ec5 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:38:27.765712 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:28.515583 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:29.284416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b4d9d-abae-4025-ab83-7a41e02d90ae service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:29.285842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-087b4d9d-abae-4025-ab83-7a41e02d90ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:29.343414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b4d9d-abae-4025-ab83-7a41e02d90ae service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:29.390780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91547fba-80d1-4ade-847c-49f82c580dd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:29.391391 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91547fba-80d1-4ade-847c-49f82c580dd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:29.391666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91547fba-80d1-4ade-847c-49f82c580dd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:29.391937 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91547fba-80d1-4ade-847c-49f82c580dd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:29.393691 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-91547fba-80d1-4ade-847c-49f82c580dd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:29.450665 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91547fba-80d1-4ade-847c-49f82c580dd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:29.452027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91547fba-80d1-4ade-847c-49f82c580dd2 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:29.461467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1386/2777] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:38:33.645752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:33.647227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:33.705382 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278de44-e43a-4fed-9e67-d62a024c0b8c admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:33.709125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee39643-2a5a-4856-96e8-79b4c897e6eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:33.710066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee39643-2a5a-4856-96e8-79b4c897e6eb service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:33.710364 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee39643-2a5a-4856-96e8-79b4c897e6eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:33.710688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee39643-2a5a-4856-96e8-79b4c897e6eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:33.712426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-aee39643-2a5a-4856-96e8-79b4c897e6eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:33.776227 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee39643-2a5a-4856-96e8-79b4c897e6eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:33.778177 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee39643-2a5a-4856-96e8-79b4c897e6eb service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:33.788833 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1392/2778] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:38:37.154930 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91547fba-80d1-4ade-847c-49f82c580dd2 service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:37.157926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-91547fba-80d1-4ade-847c-49f82c580dd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:37.238283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91547fba-80d1-4ade-847c-49f82c580dd2 service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:37.243348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4930c-ff9c-4072-9925-d79648cf1cf8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:37.243951 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4930c-ff9c-4072-9925-d79648cf1cf8 service placement] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:37.244264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4930c-ff9c-4072-9925-d79648cf1cf8 service placement] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:37.244544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4930c-ff9c-4072-9925-d79648cf1cf8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:37.246866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-31d4930c-ff9c-4072-9925-d79648cf1cf8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:37.258121 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4930c-ff9c-4072-9925-d79648cf1cf8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:37.259640 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4930c-ff9c-4072-9925-d79648cf1cf8 service placement] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:37.265557 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:37.270300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1387/2779] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:38:37.287795 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee39643-2a5a-4856-96e8-79b4c897e6eb service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:37.294272 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-aee39643-2a5a-4856-96e8-79b4c897e6eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:37.303239 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee39643-2a5a-4856-96e8-79b4c897e6eb service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:37.308563 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cd7a4-4f0e-4d5f-8cff-0e682af3236d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:37.309373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cd7a4-4f0e-4d5f-8cff-0e682af3236d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:37.309754 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cd7a4-4f0e-4d5f-8cff-0e682af3236d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:37.310166 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cd7a4-4f0e-4d5f-8cff-0e682af3236d admin admin] PATH_INFO: `/v3/users/75a2f770cc774d61baa6c07532665477` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:37.311602 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cd7a4-4f0e-4d5f-8cff-0e682af3236d admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a2f770cc774d61baa6c07532665477)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:37.313614 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cd7a4-4f0e-4d5f-8cff-0e682af3236d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:37.476409 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2a4cd7a4-4f0e-4d5f-8cff-0e682af3236d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a2f770cc774d61baa6c07532665477'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:37.483930 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2a4cd7a4-4f0e-4d5f-8cff-0e682af3236d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a2f770cc774d61baa6c07532665477'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:37.570776 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2a4cd7a4-4f0e-4d5f-8cff-0e682af3236d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a2f770cc774d61baa6c07532665477'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:37.571326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2a4cd7a4-4f0e-4d5f-8cff-0e682af3236d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a2f770cc774d61baa6c07532665477'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:37.583514 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2a4cd7a4-4f0e-4d5f-8cff-0e682af3236d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a2f770cc774d61baa6c07532665477'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:37.590005 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1393/2780] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:38:37 2022] DELETE /identity/v3/users/75a2f770cc774d61baa6c07532665477 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:38:37.869694 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4930c-ff9c-4072-9925-d79648cf1cf8 service placement] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:37.882805 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4930c-ff9c-4072-9925-d79648cf1cf8 service placement] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:37.942854 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:37.943534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:37.943818 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:37.944094 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8 admin admin] PATH_INFO: `/v3/projects/f7b499be5a424f8a860c6e30a060d941` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:37.945311 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b499be5a424f8a860c6e30a060d941)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:37.947241 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:38.117949 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b499be5a424f8a860c6e30a060d941', 'request_id': 'req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:38.123989 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b499be5a424f8a860c6e30a060d941', 'request_id': 'req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:38.237795 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b499be5a424f8a860c6e30a060d941', 'request_id': 'req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:38.250566 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8 admin admin] The token cache is being invalidate because project f7b499be5a424f8a860c6e30a060d941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:38:38.250898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:38.251515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-0bda3f23-15f2-4837-b9da-7b472ce4c5b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:38.253035 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1388/2781] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:38:37 2022] DELETE /identity/v3/projects/f7b499be5a424f8a860c6e30a060d941 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:38:38.258098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd3bd8-869f-4b85-b417-c047e127764a None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:38.258437 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd3bd8-869f-4b85-b417-c047e127764a None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:38.258716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd3bd8-869f-4b85-b417-c047e127764a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:38.278833 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-eefd3bd8-869f-4b85-b417-c047e127764a None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:38:38.280035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-eefd3bd8-869f-4b85-b417-c047e127764a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:38.296473 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:38.338829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1394/2782] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:38:38.347514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeed9a2-006c-4e6e-b0a8-0de2b16be781 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:38.347811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeed9a2-006c-4e6e-b0a8-0de2b16be781 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:38.348088 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeed9a2-006c-4e6e-b0a8-0de2b16be781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:38.367384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-cbeed9a2-006c-4e6e-b0a8-0de2b16be781 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:38:38.375510 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-cbeed9a2-006c-4e6e-b0a8-0de2b16be781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:38.377264 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1389/2783] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:38:38.381396 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd3bd8-869f-4b85-b417-c047e127764a None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:38.391394 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd3bd8-869f-4b85-b417-c047e127764a None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:38.396540 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d49a1-879f-48b3-9ed8-96b73dbc46ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:38.397141 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d49a1-879f-48b3-9ed8-96b73dbc46ad admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:38.397448 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d49a1-879f-48b3-9ed8-96b73dbc46ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:38.397725 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d49a1-879f-48b3-9ed8-96b73dbc46ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:38.398631 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d49a1-879f-48b3-9ed8-96b73dbc46ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:38:38.398957 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d49a1-879f-48b3-9ed8-96b73dbc46ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:38.400619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d49a1-879f-48b3-9ed8-96b73dbc46ad admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:38.412716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1395/2784] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:38:38.424763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeed9a2-006c-4e6e-b0a8-0de2b16be781 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:38.434411 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeed9a2-006c-4e6e-b0a8-0de2b16be781 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:38.439225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22214d-04a0-4e6f-98fb-f7ecf752adbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:38.439811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22214d-04a0-4e6f-98fb-f7ecf752adbc admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:38.440085 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22214d-04a0-4e6f-98fb-f7ecf752adbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:38.440384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22214d-04a0-4e6f-98fb-f7ecf752adbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:38.441005 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22214d-04a0-4e6f-98fb-f7ecf752adbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:38.442882 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22214d-04a0-4e6f-98fb-f7ecf752adbc admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:38.543827 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1390/2785] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:38:38 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:38:38.547204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d49a1-879f-48b3-9ed8-96b73dbc46ad admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:38.557207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d49a1-879f-48b3-9ed8-96b73dbc46ad admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:38.613081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e3aa0-bbce-4117-a155-41c727cdb7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:38.613690 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e3aa0-bbce-4117-a155-41c727cdb7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:38.614094 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e3aa0-bbce-4117-a155-41c727cdb7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:38.614378 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e3aa0-bbce-4117-a155-41c727cdb7f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:38.615074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e3aa0-bbce-4117-a155-41c727cdb7f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:38.616961 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e3aa0-bbce-4117-a155-41c727cdb7f2 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:38.706478 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1396/2786] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:38:38 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:38:38.710607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22214d-04a0-4e6f-98fb-f7ecf752adbc admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:38.720890 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22214d-04a0-4e6f-98fb-f7ecf752adbc admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:38.725459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcade3e-010f-40e1-a366-244683bac19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:38.726040 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcade3e-010f-40e1-a366-244683bac19f admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:38.726318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcade3e-010f-40e1-a366-244683bac19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:38.726588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcade3e-010f-40e1-a366-244683bac19f admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:38.727221 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcade3e-010f-40e1-a366-244683bac19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:38.728840 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcade3e-010f-40e1-a366-244683bac19f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:38.739226 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1391/2787] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:38:38.743383 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e3aa0-bbce-4117-a155-41c727cdb7f2 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:38.754505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e3aa0-bbce-4117-a155-41c727cdb7f2 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:38.759477 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989203e8-d99f-430a-b2c1-9bb8b3e4427a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:38.760065 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989203e8-d99f-430a-b2c1-9bb8b3e4427a admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:38.760440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989203e8-d99f-430a-b2c1-9bb8b3e4427a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:38.760714 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989203e8-d99f-430a-b2c1-9bb8b3e4427a admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:38.761368 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989203e8-d99f-430a-b2c1-9bb8b3e4427a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:38.762956 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989203e8-d99f-430a-b2c1-9bb8b3e4427a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:38.774232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1397/2788] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:38:38.778453 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcade3e-010f-40e1-a366-244683bac19f admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:38.788446 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcade3e-010f-40e1-a366-244683bac19f admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:38.793591 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee0e38-2ab0-41f8-a59e-01f6537896fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:38.794252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee0e38-2ab0-41f8-a59e-01f6537896fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:38.794515 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee0e38-2ab0-41f8-a59e-01f6537896fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:38.794792 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee0e38-2ab0-41f8-a59e-01f6537896fa admin admin] PATH_INFO: `/v3/projects/72eda801bca64d9f91a2a48262a36a1f/users/e5a3df08e75f4861a068bae545e42b45/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:38.809714 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee0e38-2ab0-41f8-a59e-01f6537896fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=72eda801bca64d9f91a2a48262a36a1f, user_id=e5a3df08e75f4861a068bae545e42b45, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:38.812390 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee0e38-2ab0-41f8-a59e-01f6537896fa admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:38.885481 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1392/2789] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:38:38 2022] PUT /identity/v3/projects/72eda801bca64d9f91a2a48262a36a1f/users/e5a3df08e75f4861a068bae545e42b45/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:38:38.895641 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b549ecb-4cef-4e90-b995-7afe00b2acb1 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:38.895935 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b549ecb-4cef-4e90-b995-7afe00b2acb1 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:38.896245 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b549ecb-4cef-4e90-b995-7afe00b2acb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:38.924640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-9b549ecb-4cef-4e90-b995-7afe00b2acb1 None None] MFA Rules not processed for user `e5a3df08e75f4861a068bae545e42b45`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:38:38.925906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9b549ecb-4cef-4e90-b995-7afe00b2acb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:38.991426 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1398/2790] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:38:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:38:39.002809 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee0e38-2ab0-41f8-a59e-01f6537896fa admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:39.004850 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-69ee0e38-2ab0-41f8-a59e-01f6537896fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:39.037754 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:39.081073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee0e38-2ab0-41f8-a59e-01f6537896fa admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:39.086927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826044e-fd9a-48de-a70d-f8d152e6529e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:39.087573 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826044e-fd9a-48de-a70d-f8d152e6529e service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:39.087855 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826044e-fd9a-48de-a70d-f8d152e6529e service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:39.088128 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826044e-fd9a-48de-a70d-f8d152e6529e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:39.096596 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826044e-fd9a-48de-a70d-f8d152e6529e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:39.098185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826044e-fd9a-48de-a70d-f8d152e6529e service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:39.107314 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1393/2791] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:38:39.475421 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b549ecb-4cef-4e90-b995-7afe00b2acb1 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:39.494893 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:39.538763 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b549ecb-4cef-4e90-b995-7afe00b2acb1 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:39.543240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:39.543850 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:39.544134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:39.544440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] PATH_INFO: `/v3/users/e5a3df08e75f4861a068bae545e42b45` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:39.545549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5a3df08e75f4861a068bae545e42b45)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:39.547518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:39.751439 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5a3df08e75f4861a068bae545e42b45'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:39.758155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5a3df08e75f4861a068bae545e42b45'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:39.862060 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5a3df08e75f4861a068bae545e42b45'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:39.862622 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5a3df08e75f4861a068bae545e42b45'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:39.875375 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5a3df08e75f4861a068bae545e42b45'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:39.886811 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1399/2792] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:38:39 2022] DELETE /identity/v3/users/e5a3df08e75f4861a068bae545e42b45 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:38:39.886961 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fa784-2c40-4d74-8d18-7fe014b9d3c8 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:39.886961 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fa784-2c40-4d74-8d18-7fe014b9d3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:39.886961 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fa784-2c40-4d74-8d18-7fe014b9d3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:39.905392 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-5b5fa784-2c40-4d74-8d18-7fe014b9d3c8 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:38:39.906586 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5b5fa784-2c40-4d74-8d18-7fe014b9d3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:39.964076 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1394/2793] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:38:39.973976 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:39.975479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:40.035917 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be326de-8abd-45ee-be51-81f37a6f6ecf admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:40.039558 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66c9bb-9373-42ba-9fcc-c3a233fa9e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:40.040189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66c9bb-9373-42ba-9fcc-c3a233fa9e05 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:40.040505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66c9bb-9373-42ba-9fcc-c3a233fa9e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:40.040796 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66c9bb-9373-42ba-9fcc-c3a233fa9e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:40.050194 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66c9bb-9373-42ba-9fcc-c3a233fa9e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:40.051612 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66c9bb-9373-42ba-9fcc-c3a233fa9e05 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:40.062014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1400/2794] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:38:40.584095 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fa784-2c40-4d74-8d18-7fe014b9d3c8 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:40.585555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5b5fa784-2c40-4d74-8d18-7fe014b9d3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:40.597865 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fa784-2c40-4d74-8d18-7fe014b9d3c8 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:40.602768 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56798ac2-785f-430a-b7ef-2ab87411fa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:40.603474 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56798ac2-785f-430a-b7ef-2ab87411fa8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:40.615723 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56798ac2-785f-430a-b7ef-2ab87411fa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:40.615723 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56798ac2-785f-430a-b7ef-2ab87411fa8b admin admin] PATH_INFO: `/v3/projects/72eda801bca64d9f91a2a48262a36a1f` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:40.615723 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56798ac2-785f-430a-b7ef-2ab87411fa8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=72eda801bca64d9f91a2a48262a36a1f)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:40.615723 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56798ac2-785f-430a-b7ef-2ab87411fa8b admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:40.755048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-56798ac2-785f-430a-b7ef-2ab87411fa8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72eda801bca64d9f91a2a48262a36a1f', 'request_id': 'req-56798ac2-785f-430a-b7ef-2ab87411fa8b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:40.760880 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-56798ac2-785f-430a-b7ef-2ab87411fa8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72eda801bca64d9f91a2a48262a36a1f', 'request_id': 'req-56798ac2-785f-430a-b7ef-2ab87411fa8b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:40.826203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-56798ac2-785f-430a-b7ef-2ab87411fa8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72eda801bca64d9f91a2a48262a36a1f', 'request_id': 'req-56798ac2-785f-430a-b7ef-2ab87411fa8b'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:40.837590 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-56798ac2-785f-430a-b7ef-2ab87411fa8b admin admin] The token cache is being invalidate because project 72eda801bca64d9f91a2a48262a36a1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:38:40.837828 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-56798ac2-785f-430a-b7ef-2ab87411fa8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:40.838436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-56798ac2-785f-430a-b7ef-2ab87411fa8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:40.839935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1395/2795] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:38:40 2022] DELETE /identity/v3/projects/72eda801bca64d9f91a2a48262a36a1f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:38:40.844765 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21cd3a-8281-4fb4-9e27-a92ce4fab360 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:40.845355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21cd3a-8281-4fb4-9e27-a92ce4fab360 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:40.845355 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21cd3a-8281-4fb4-9e27-a92ce4fab360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:40.865098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-fc21cd3a-8281-4fb4-9e27-a92ce4fab360 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:38:40.866273 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-fc21cd3a-8281-4fb4-9e27-a92ce4fab360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:40.903664 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:40.929438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1401/2796] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:38:40.941607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a8ee0-2708-4d6f-a4eb-476db12a9d1c None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:40.942109 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a8ee0-2708-4d6f-a4eb-476db12a9d1c None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:40.942209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a8ee0-2708-4d6f-a4eb-476db12a9d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:40.961798 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-487a8ee0-2708-4d6f-a4eb-476db12a9d1c None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:38:40.963012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-487a8ee0-2708-4d6f-a4eb-476db12a9d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:40.970834 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1396/2797] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:38:40.975172 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21cd3a-8281-4fb4-9e27-a92ce4fab360 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:40.985663 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21cd3a-8281-4fb4-9e27-a92ce4fab360 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:40.990364 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a203bce-a290-4f71-972c-bb0fe3df0e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:40.996034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a203bce-a290-4f71-972c-bb0fe3df0e59 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:40.996034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a203bce-a290-4f71-972c-bb0fe3df0e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:40.996034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a203bce-a290-4f71-972c-bb0fe3df0e59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:40.996034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a203bce-a290-4f71-972c-bb0fe3df0e59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:38:40.996034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a203bce-a290-4f71-972c-bb0fe3df0e59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:40.996034 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a203bce-a290-4f71-972c-bb0fe3df0e59 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:41.005369 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1402/2798] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:38:41.009475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a8ee0-2708-4d6f-a4eb-476db12a9d1c None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:41.019327 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a8ee0-2708-4d6f-a4eb-476db12a9d1c None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:41.024325 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e523ed-7a13-4987-bbe7-b289f01faeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:41.024913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e523ed-7a13-4987-bbe7-b289f01faeb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:41.025219 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e523ed-7a13-4987-bbe7-b289f01faeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:41.025494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e523ed-7a13-4987-bbe7-b289f01faeb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:41.026152 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e523ed-7a13-4987-bbe7-b289f01faeb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:41.028675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e523ed-7a13-4987-bbe7-b289f01faeb8 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:41.113434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1397/2799] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:38:41 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:38:41.117460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a203bce-a290-4f71-972c-bb0fe3df0e59 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:41.128127 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a203bce-a290-4f71-972c-bb0fe3df0e59 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:41.183718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda9a9b-3a7a-4dd1-9a06-9a290ce08d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:41.184341 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda9a9b-3a7a-4dd1-9a06-9a290ce08d4b admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:41.184620 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda9a9b-3a7a-4dd1-9a06-9a290ce08d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:41.184886 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda9a9b-3a7a-4dd1-9a06-9a290ce08d4b admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:41.185515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda9a9b-3a7a-4dd1-9a06-9a290ce08d4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:41.187414 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda9a9b-3a7a-4dd1-9a06-9a290ce08d4b admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:41.313604 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1403/2800] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:38:41 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:38:41.317619 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e523ed-7a13-4987-bbe7-b289f01faeb8 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:41.328002 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e523ed-7a13-4987-bbe7-b289f01faeb8 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:41.332695 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369fed2-59d7-4a5a-b65a-804bbd027f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:41.333291 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369fed2-59d7-4a5a-b65a-804bbd027f36 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:41.333572 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369fed2-59d7-4a5a-b65a-804bbd027f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:41.333847 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369fed2-59d7-4a5a-b65a-804bbd027f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:41.334489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369fed2-59d7-4a5a-b65a-804bbd027f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:41.336069 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369fed2-59d7-4a5a-b65a-804bbd027f36 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:41.337310 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:41.346990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1398/2801] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:38:41.350849 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda9a9b-3a7a-4dd1-9a06-9a290ce08d4b admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:41.361414 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda9a9b-3a7a-4dd1-9a06-9a290ce08d4b admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:41.365973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d198b9-adc9-42e9-9147-4f4e6e1f8a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:41.366550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d198b9-adc9-42e9-9147-4f4e6e1f8a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:41.366828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d198b9-adc9-42e9-9147-4f4e6e1f8a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:41.367098 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d198b9-adc9-42e9-9147-4f4e6e1f8a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:41.367754 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d198b9-adc9-42e9-9147-4f4e6e1f8a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:41.369403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d198b9-adc9-42e9-9147-4f4e6e1f8a66 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:41.379739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1404/2802] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:38:41.383700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369fed2-59d7-4a5a-b65a-804bbd027f36 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:41.393399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369fed2-59d7-4a5a-b65a-804bbd027f36 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:41.397935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b73c7-b7a7-47e1-9d02-57991dd22474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:41.398851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b73c7-b7a7-47e1-9d02-57991dd22474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:41.399421 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b73c7-b7a7-47e1-9d02-57991dd22474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:41.399945 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b73c7-b7a7-47e1-9d02-57991dd22474 admin admin] PATH_INFO: `/v3/projects/5f12df4ea0754958b17a679a2cd3f648/users/859562b79fc04a7e986314dba51d798b/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:41.416541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b73c7-b7a7-47e1-9d02-57991dd22474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f12df4ea0754958b17a679a2cd3f648, user_id=859562b79fc04a7e986314dba51d798b, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:41.420635 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b73c7-b7a7-47e1-9d02-57991dd22474 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:41.514521 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1399/2803] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:38:41 2022] PUT /identity/v3/projects/5f12df4ea0754958b17a679a2cd3f648/users/859562b79fc04a7e986314dba51d798b/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:38:41.526562 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29140735-a9b6-4023-9c3c-719e153e0ecb None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:41.526847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29140735-a9b6-4023-9c3c-719e153e0ecb None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:41.527123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29140735-a9b6-4023-9c3c-719e153e0ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:41.556642 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-29140735-a9b6-4023-9c3c-719e153e0ecb None None] MFA Rules not processed for user `859562b79fc04a7e986314dba51d798b`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:38:41.557837 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-29140735-a9b6-4023-9c3c-719e153e0ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:41.624126 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1405/2804] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:38:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:38:41.635238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b73c7-b7a7-47e1-9d02-57991dd22474 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:41.636975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-427b73c7-b7a7-47e1-9d02-57991dd22474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:41.693676 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b73c7-b7a7-47e1-9d02-57991dd22474 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:41.697319 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55113b2d-6f8f-46be-853e-33da07431b03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:41.697909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55113b2d-6f8f-46be-853e-33da07431b03 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:41.698185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55113b2d-6f8f-46be-853e-33da07431b03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:41.698486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55113b2d-6f8f-46be-853e-33da07431b03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:41.707569 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55113b2d-6f8f-46be-853e-33da07431b03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:41.709197 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55113b2d-6f8f-46be-853e-33da07431b03 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:41.718460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1400/2805] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:38:51.621267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:51.713931 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:07.113844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29140735-a9b6-4023-9c3c-719e153e0ecb None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:07.175546 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29140735-a9b6-4023-9c3c-719e153e0ecb None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:07.180074 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:07.180726 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:07.181008 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:07.181280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] PATH_INFO: `/v3/users/859562b79fc04a7e986314dba51d798b` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:07.182414 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=859562b79fc04a7e986314dba51d798b)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:07.184377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:07.377949 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '859562b79fc04a7e986314dba51d798b'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:07.384791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '859562b79fc04a7e986314dba51d798b'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:07.462079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '859562b79fc04a7e986314dba51d798b'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:07.462717 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '859562b79fc04a7e986314dba51d798b'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:07.475157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '859562b79fc04a7e986314dba51d798b'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:07.480892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1406/2806] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:07 2022] DELETE /identity/v3/users/859562b79fc04a7e986314dba51d798b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:07.486167 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc4191-b0c5-4df9-a2af-f2eac34080f4 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:07.486692 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc4191-b0c5-4df9-a2af-f2eac34080f4 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:07.486803 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc4191-b0c5-4df9-a2af-f2eac34080f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:07.505683 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-50cc4191-b0c5-4df9-a2af-f2eac34080f4 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:07.506869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-50cc4191-b0c5-4df9-a2af-f2eac34080f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:07.564287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1401/2807] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:07.574089 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:07.575575 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:07.634251 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d345e2-8f40-4459-a2c3-5dd72ab22fe5 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:07.637870 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef756c7-21da-4cfe-99ce-9e548027a323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:07.638469 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef756c7-21da-4cfe-99ce-9e548027a323 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:07.639374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef756c7-21da-4cfe-99ce-9e548027a323 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:07.639692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef756c7-21da-4cfe-99ce-9e548027a323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:07.648800 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef756c7-21da-4cfe-99ce-9e548027a323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:07.650226 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef756c7-21da-4cfe-99ce-9e548027a323 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:07.659525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1407/2808] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:08.063511 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc4191-b0c5-4df9-a2af-f2eac34080f4 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:08.064958 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-50cc4191-b0c5-4df9-a2af-f2eac34080f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:08.078496 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc4191-b0c5-4df9-a2af-f2eac34080f4 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:08.084638 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af64ca-2722-4b18-ba64-813b1cbdc24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:08.085436 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af64ca-2722-4b18-ba64-813b1cbdc24c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.085720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af64ca-2722-4b18-ba64-813b1cbdc24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.086017 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af64ca-2722-4b18-ba64-813b1cbdc24c admin admin] PATH_INFO: `/v3/projects/5f12df4ea0754958b17a679a2cd3f648` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.087165 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af64ca-2722-4b18-ba64-813b1cbdc24c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f12df4ea0754958b17a679a2cd3f648)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:08.089207 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af64ca-2722-4b18-ba64-813b1cbdc24c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:08.215858 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-21af64ca-2722-4b18-ba64-813b1cbdc24c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f12df4ea0754958b17a679a2cd3f648', 'request_id': 'req-21af64ca-2722-4b18-ba64-813b1cbdc24c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.221727 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-21af64ca-2722-4b18-ba64-813b1cbdc24c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f12df4ea0754958b17a679a2cd3f648', 'request_id': 'req-21af64ca-2722-4b18-ba64-813b1cbdc24c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.313193 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-21af64ca-2722-4b18-ba64-813b1cbdc24c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f12df4ea0754958b17a679a2cd3f648', 'request_id': 'req-21af64ca-2722-4b18-ba64-813b1cbdc24c'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.324183 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-21af64ca-2722-4b18-ba64-813b1cbdc24c admin admin] The token cache is being invalidate because project 5f12df4ea0754958b17a679a2cd3f648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:08.324492 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-21af64ca-2722-4b18-ba64-813b1cbdc24c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.325118 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-21af64ca-2722-4b18-ba64-813b1cbdc24c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.326601 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1402/2809] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:08 2022] DELETE /identity/v3/projects/5f12df4ea0754958b17a679a2cd3f648 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:08.331794 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c3904-7108-4abc-a03d-eca3ac888bbd None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.332297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c3904-7108-4abc-a03d-eca3ac888bbd None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.332408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c3904-7108-4abc-a03d-eca3ac888bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.352359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-876c3904-7108-4abc-a03d-eca3ac888bbd None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:08.353557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-876c3904-7108-4abc-a03d-eca3ac888bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:08.370906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:08.411899 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1408/2810] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:08.419533 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f3c9a-73a0-4818-9202-105d94117d29 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.419854 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f3c9a-73a0-4818-9202-105d94117d29 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.420220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f3c9a-73a0-4818-9202-105d94117d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.439343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-3a8f3c9a-73a0-4818-9202-105d94117d29 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:08.440565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3a8f3c9a-73a0-4818-9202-105d94117d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:08.448282 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1403/2811] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:08.452896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c3904-7108-4abc-a03d-eca3ac888bbd None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:08.462586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c3904-7108-4abc-a03d-eca3ac888bbd None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:08.467822 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0245677-6401-4b88-9014-a2b0f858f94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:08.468403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0245677-6401-4b88-9014-a2b0f858f94a admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.468683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0245677-6401-4b88-9014-a2b0f858f94a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.468955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0245677-6401-4b88-9014-a2b0f858f94a admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.469603 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0245677-6401-4b88-9014-a2b0f858f94a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:39:08.469908 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0245677-6401-4b88-9014-a2b0f858f94a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:08.471530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0245677-6401-4b88-9014-a2b0f858f94a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:08.483207 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1409/2812] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:39:08.487157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f3c9a-73a0-4818-9202-105d94117d29 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:08.496467 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f3c9a-73a0-4818-9202-105d94117d29 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:08.501233 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35b008-4035-4f50-9cdd-d727b1a34d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:08.501822 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35b008-4035-4f50-9cdd-d727b1a34d16 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.502104 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35b008-4035-4f50-9cdd-d727b1a34d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.502394 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35b008-4035-4f50-9cdd-d727b1a34d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.503027 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35b008-4035-4f50-9cdd-d727b1a34d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:08.504953 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35b008-4035-4f50-9cdd-d727b1a34d16 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:08.583509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1404/2813] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:08 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:08.587839 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0245677-6401-4b88-9014-a2b0f858f94a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:08.598665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0245677-6401-4b88-9014-a2b0f858f94a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:08.656360 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7327b0c-2d22-40b3-baf8-6920bc6bf2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:08.656955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7327b0c-2d22-40b3-baf8-6920bc6bf2cb admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.657237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7327b0c-2d22-40b3-baf8-6920bc6bf2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.657512 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7327b0c-2d22-40b3-baf8-6920bc6bf2cb admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.658136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7327b0c-2d22-40b3-baf8-6920bc6bf2cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:08.660038 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7327b0c-2d22-40b3-baf8-6920bc6bf2cb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:08.788659 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1410/2814] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:08 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:08.792724 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35b008-4035-4f50-9cdd-d727b1a34d16 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:08.802851 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35b008-4035-4f50-9cdd-d727b1a34d16 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:08.807696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7facd2-e487-4437-8fe5-48980d6f90f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:08.808346 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7facd2-e487-4437-8fe5-48980d6f90f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.808627 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7facd2-e487-4437-8fe5-48980d6f90f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.808901 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7facd2-e487-4437-8fe5-48980d6f90f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.809538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7facd2-e487-4437-8fe5-48980d6f90f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:08.811661 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7facd2-e487-4437-8fe5-48980d6f90f0 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:08.821565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1405/2815] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:08.825211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7327b0c-2d22-40b3-baf8-6920bc6bf2cb admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:08.835697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7327b0c-2d22-40b3-baf8-6920bc6bf2cb admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:08.840343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f97ca9-cfa2-48f5-bf99-d3c1d44766dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:08.840918 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f97ca9-cfa2-48f5-bf99-d3c1d44766dc admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.841193 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f97ca9-cfa2-48f5-bf99-d3c1d44766dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.841467 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f97ca9-cfa2-48f5-bf99-d3c1d44766dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.842144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f97ca9-cfa2-48f5-bf99-d3c1d44766dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:08.843712 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f97ca9-cfa2-48f5-bf99-d3c1d44766dc admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:08.854338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1411/2816] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:08.858371 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7facd2-e487-4437-8fe5-48980d6f90f0 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:08.868435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7facd2-e487-4437-8fe5-48980d6f90f0 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:08.873538 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2686aed-92d1-44fa-8d45-d6755a3cb226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:08.874178 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2686aed-92d1-44fa-8d45-d6755a3cb226 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.874460 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2686aed-92d1-44fa-8d45-d6755a3cb226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.874736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2686aed-92d1-44fa-8d45-d6755a3cb226 admin admin] PATH_INFO: `/v3/projects/41dd1bfbf447449eb4accf9506fa9388/users/8df3b28ecead401b88fc0418e235fcb0/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.889770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2686aed-92d1-44fa-8d45-d6755a3cb226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41dd1bfbf447449eb4accf9506fa9388, user_id=8df3b28ecead401b88fc0418e235fcb0, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:08.892435 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2686aed-92d1-44fa-8d45-d6755a3cb226 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:08.999494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1406/2817] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:08 2022] PUT /identity/v3/projects/41dd1bfbf447449eb4accf9506fa9388/users/8df3b28ecead401b88fc0418e235fcb0/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:09.010924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c7d20-db90-4485-9fca-06b33188f7b8 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.011214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c7d20-db90-4485-9fca-06b33188f7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.011485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c7d20-db90-4485-9fca-06b33188f7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.039573 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-023c7d20-db90-4485-9fca-06b33188f7b8 None None] MFA Rules not processed for user `8df3b28ecead401b88fc0418e235fcb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:09.040739 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-023c7d20-db90-4485-9fca-06b33188f7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:09.104075 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1412/2818] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:09.114790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2686aed-92d1-44fa-8d45-d6755a3cb226 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:09.116486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f2686aed-92d1-44fa-8d45-d6755a3cb226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:09.139450 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:09.173983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2686aed-92d1-44fa-8d45-d6755a3cb226 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:09.177643 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3c37c-a448-48e8-8fa1-266cd1b135e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:09.178238 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3c37c-a448-48e8-8fa1-266cd1b135e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.178518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3c37c-a448-48e8-8fa1-266cd1b135e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.178849 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3c37c-a448-48e8-8fa1-266cd1b135e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.187899 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3c37c-a448-48e8-8fa1-266cd1b135e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:09.189363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3c37c-a448-48e8-8fa1-266cd1b135e6 service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:09.197910 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1407/2819] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:13.516884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c7d20-db90-4485-9fca-06b33188f7b8 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:13.536611 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:13.581068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c7d20-db90-4485-9fca-06b33188f7b8 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:13.585697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:13.586343 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:13.586626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:13.586902 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] PATH_INFO: `/v3/users/8df3b28ecead401b88fc0418e235fcb0` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:13.588040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df3b28ecead401b88fc0418e235fcb0)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:13.590025 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:13.899569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df3b28ecead401b88fc0418e235fcb0'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:13.907155 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df3b28ecead401b88fc0418e235fcb0'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:14.016431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df3b28ecead401b88fc0418e235fcb0'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:14.017157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df3b28ecead401b88fc0418e235fcb0'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:14.031777 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df3b28ecead401b88fc0418e235fcb0'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:14.039106 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1413/2820] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:13 2022] DELETE /identity/v3/users/8df3b28ecead401b88fc0418e235fcb0 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:14.047352 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b7898-58ab-44cb-8aa7-9e629cb8badd None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:14.047869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b7898-58ab-44cb-8aa7-9e629cb8badd None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:14.047972 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b7898-58ab-44cb-8aa7-9e629cb8badd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:14.069028 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-855b7898-58ab-44cb-8aa7-9e629cb8badd None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:14.070252 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-855b7898-58ab-44cb-8aa7-9e629cb8badd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:14.133391 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1408/2821] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:14.144040 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:14.145521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:14.206903 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e32e30-ac99-43ff-a4d5-08174d5f18dd admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:14.210650 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48e3b1-b7f8-4a1d-8acb-5c809332e471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:14.211245 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48e3b1-b7f8-4a1d-8acb-5c809332e471 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:14.211522 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48e3b1-b7f8-4a1d-8acb-5c809332e471 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:14.211800 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48e3b1-b7f8-4a1d-8acb-5c809332e471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:14.220969 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48e3b1-b7f8-4a1d-8acb-5c809332e471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:14.223324 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48e3b1-b7f8-4a1d-8acb-5c809332e471 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:14.233099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1414/2822] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:14.687811 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b7898-58ab-44cb-8aa7-9e629cb8badd None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:14.689304 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-855b7898-58ab-44cb-8aa7-9e629cb8badd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:14.701995 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b7898-58ab-44cb-8aa7-9e629cb8badd None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:14.710043 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5a796-b98b-4e3d-ac81-f0a0211129e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:14.710618 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5a796-b98b-4e3d-ac81-f0a0211129e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:14.710898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5a796-b98b-4e3d-ac81-f0a0211129e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:14.711173 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5a796-b98b-4e3d-ac81-f0a0211129e7 admin admin] PATH_INFO: `/v3/projects/41dd1bfbf447449eb4accf9506fa9388` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:14.712382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5a796-b98b-4e3d-ac81-f0a0211129e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41dd1bfbf447449eb4accf9506fa9388)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:14.714544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5a796-b98b-4e3d-ac81-f0a0211129e7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:14.885722 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-74e5a796-b98b-4e3d-ac81-f0a0211129e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41dd1bfbf447449eb4accf9506fa9388', 'request_id': 'req-74e5a796-b98b-4e3d-ac81-f0a0211129e7'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:14.891550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-74e5a796-b98b-4e3d-ac81-f0a0211129e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41dd1bfbf447449eb4accf9506fa9388', 'request_id': 'req-74e5a796-b98b-4e3d-ac81-f0a0211129e7'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:15.005962 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-74e5a796-b98b-4e3d-ac81-f0a0211129e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41dd1bfbf447449eb4accf9506fa9388', 'request_id': 'req-74e5a796-b98b-4e3d-ac81-f0a0211129e7'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:15.020484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-74e5a796-b98b-4e3d-ac81-f0a0211129e7 admin admin] The token cache is being invalidate because project 41dd1bfbf447449eb4accf9506fa9388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81859) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:15.020776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-74e5a796-b98b-4e3d-ac81-f0a0211129e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:15.021414 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-74e5a796-b98b-4e3d-ac81-f0a0211129e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:15.022895 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1409/2823] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:14 2022] DELETE /identity/v3/projects/41dd1bfbf447449eb4accf9506fa9388 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:15.028142 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d130a-bd1e-401b-8b14-c0435f89b69d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:15.028672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d130a-bd1e-401b-8b14-c0435f89b69d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:15.028672 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d130a-bd1e-401b-8b14-c0435f89b69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:15.048969 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-920d130a-bd1e-401b-8b14-c0435f89b69d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:15.050194 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-920d130a-bd1e-401b-8b14-c0435f89b69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:15.086247 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:15.114053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1415/2824] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:15.121980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383320fc-d9a0-4233-855b-0fae99d18d3f None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:15.122487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383320fc-d9a0-4233-855b-0fae99d18d3f None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:15.122487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383320fc-d9a0-4233-855b-0fae99d18d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:15.142337 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-383320fc-d9a0-4233-855b-0fae99d18d3f None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:15.143525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-383320fc-d9a0-4233-855b-0fae99d18d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:15.151909 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1410/2825] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:15.161866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d130a-bd1e-401b-8b14-c0435f89b69d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:15.171447 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d130a-bd1e-401b-8b14-c0435f89b69d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:15.176024 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371884e3-3ecd-47a8-a31f-cc17e908ef3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:15.176645 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371884e3-3ecd-47a8-a31f-cc17e908ef3e admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:15.176925 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371884e3-3ecd-47a8-a31f-cc17e908ef3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:15.177199 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371884e3-3ecd-47a8-a31f-cc17e908ef3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:15.177871 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371884e3-3ecd-47a8-a31f-cc17e908ef3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81858) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:39:15.178200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371884e3-3ecd-47a8-a31f-cc17e908ef3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:15.179779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371884e3-3ecd-47a8-a31f-cc17e908ef3e admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:15.191446 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1416/2826] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:39:15.195741 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383320fc-d9a0-4233-855b-0fae99d18d3f None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:15.205655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383320fc-d9a0-4233-855b-0fae99d18d3f None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:15.210348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541115e7-d876-4277-93f6-89fdf72e6c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:15.210935 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541115e7-d876-4277-93f6-89fdf72e6c51 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:15.211216 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541115e7-d876-4277-93f6-89fdf72e6c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:15.211487 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541115e7-d876-4277-93f6-89fdf72e6c51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:15.212120 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541115e7-d876-4277-93f6-89fdf72e6c51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:15.214042 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541115e7-d876-4277-93f6-89fdf72e6c51 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:15.313204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1411/2827] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:15 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:15.317497 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371884e3-3ecd-47a8-a31f-cc17e908ef3e admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:15.328204 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371884e3-3ecd-47a8-a31f-cc17e908ef3e admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:15.383173 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f677796-d62f-4798-a7aa-be68c5eae2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:15.383767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f677796-d62f-4798-a7aa-be68c5eae2ea admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:15.384049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f677796-d62f-4798-a7aa-be68c5eae2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:15.384357 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f677796-d62f-4798-a7aa-be68c5eae2ea admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:15.384986 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f677796-d62f-4798-a7aa-be68c5eae2ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:15.386922 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f677796-d62f-4798-a7aa-be68c5eae2ea admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:15.559166 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1417/2828] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:15 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:15.572825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541115e7-d876-4277-93f6-89fdf72e6c51 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:15.573846 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541115e7-d876-4277-93f6-89fdf72e6c51 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:15.578432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892ebd4-0eae-4430-a974-4abe893d48ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:15.579021 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892ebd4-0eae-4430-a974-4abe893d48ea admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:15.579315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892ebd4-0eae-4430-a974-4abe893d48ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:15.579581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892ebd4-0eae-4430-a974-4abe893d48ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:15.580258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892ebd4-0eae-4430-a974-4abe893d48ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:15.581881 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892ebd4-0eae-4430-a974-4abe893d48ea admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:15.592307 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1412/2829] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:15.596351 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f677796-d62f-4798-a7aa-be68c5eae2ea admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:15.606597 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f677796-d62f-4798-a7aa-be68c5eae2ea admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:15.612017 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04002073-0127-43eb-9c88-d11da3be2f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:15.612694 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04002073-0127-43eb-9c88-d11da3be2f86 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:15.612973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04002073-0127-43eb-9c88-d11da3be2f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:15.613243 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04002073-0127-43eb-9c88-d11da3be2f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:15.613891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04002073-0127-43eb-9c88-d11da3be2f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:15.615485 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04002073-0127-43eb-9c88-d11da3be2f86 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:15.626609 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1418/2830] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:15.631391 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892ebd4-0eae-4430-a974-4abe893d48ea admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:15.641514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892ebd4-0eae-4430-a974-4abe893d48ea admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:15.646101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd000f-a326-406e-8587-8f840df2f7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:15.646731 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd000f-a326-406e-8587-8f840df2f7e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:15.647073 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd000f-a326-406e-8587-8f840df2f7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:15.647345 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd000f-a326-406e-8587-8f840df2f7e3 admin admin] PATH_INFO: `/v3/projects/242e747402e04bfdad34c63223d806a5/users/8a1ffec6f7254fcc9e750a50f2e650a6/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:15.649814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:15.663384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd000f-a326-406e-8587-8f840df2f7e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=242e747402e04bfdad34c63223d806a5, user_id=8a1ffec6f7254fcc9e750a50f2e650a6, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:15.667585 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd000f-a326-406e-8587-8f840df2f7e3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:15.774679 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1413/2831] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:15 2022] PUT /identity/v3/projects/242e747402e04bfdad34c63223d806a5/users/8a1ffec6f7254fcc9e750a50f2e650a6/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:15.785100 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787c43f-d15a-4848-bf2a-9dd940f20b0b None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:15.785379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787c43f-d15a-4848-bf2a-9dd940f20b0b None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:15.785646 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787c43f-d15a-4848-bf2a-9dd940f20b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:15.814459 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e787c43f-d15a-4848-bf2a-9dd940f20b0b None None] MFA Rules not processed for user `8a1ffec6f7254fcc9e750a50f2e650a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:15.815701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e787c43f-d15a-4848-bf2a-9dd940f20b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:15.882744 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1419/2832] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:15.887360 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd000f-a326-406e-8587-8f840df2f7e3 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:15.949721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd000f-a326-406e-8587-8f840df2f7e3 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:15.955338 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe9b4f-ba7c-47b4-be51-2639cb8cddaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:15.955936 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe9b4f-ba7c-47b4-be51-2639cb8cddaa admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:15.956245 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe9b4f-ba7c-47b4-be51-2639cb8cddaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:15.956524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe9b4f-ba7c-47b4-be51-2639cb8cddaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:15.957149 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe9b4f-ba7c-47b4-be51-2639cb8cddaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:15.959084 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe9b4f-ba7c-47b4-be51-2639cb8cddaa admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:16.178358 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1414/2833] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:15 2022] POST /identity/v3/projects => generated 385 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:16.182628 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787c43f-d15a-4848-bf2a-9dd940f20b0b None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:16.194014 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787c43f-d15a-4848-bf2a-9dd940f20b0b None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:16.225706 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:16.250353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a069ec-cff2-447d-957f-cf42570887f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:16.258979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a069ec-cff2-447d-957f-cf42570887f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:16.258979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a069ec-cff2-447d-957f-cf42570887f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:16.258979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a069ec-cff2-447d-957f-cf42570887f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:16.258979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a069ec-cff2-447d-957f-cf42570887f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:16.258979 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a069ec-cff2-447d-957f-cf42570887f3 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:16.393862 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1420/2834] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:16 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:16.398208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe9b4f-ba7c-47b4-be51-2639cb8cddaa admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:16.409484 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe9b4f-ba7c-47b4-be51-2639cb8cddaa admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:16.414102 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42d04d-e90d-45ad-bb65-46d67a5fc8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:16.414713 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42d04d-e90d-45ad-bb65-46d67a5fc8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:16.414996 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42d04d-e90d-45ad-bb65-46d67a5fc8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:16.415274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42d04d-e90d-45ad-bb65-46d67a5fc8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:16.415999 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42d04d-e90d-45ad-bb65-46d67a5fc8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:16.417632 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42d04d-e90d-45ad-bb65-46d67a5fc8d3 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:16.429405 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1415/2835] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:16.433262 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a069ec-cff2-447d-957f-cf42570887f3 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:16.443491 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a069ec-cff2-447d-957f-cf42570887f3 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:16.448408 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83476468-0c76-443d-9d26-b4a77930246a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:16.449068 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476468-0c76-443d-9d26-b4a77930246a admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:16.456774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476468-0c76-443d-9d26-b4a77930246a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:16.456774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476468-0c76-443d-9d26-b4a77930246a admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:16.456774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83476468-0c76-443d-9d26-b4a77930246a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:16.456774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83476468-0c76-443d-9d26-b4a77930246a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:16.462089 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1421/2836] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:16.466998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42d04d-e90d-45ad-bb65-46d67a5fc8d3 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:16.477115 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42d04d-e90d-45ad-bb65-46d67a5fc8d3 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:16.481762 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6a3e4-392f-4f93-b3bb-1a0586b7ba16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:16.483048 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6a3e4-392f-4f93-b3bb-1a0586b7ba16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:16.483336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6a3e4-392f-4f93-b3bb-1a0586b7ba16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:16.483607 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6a3e4-392f-4f93-b3bb-1a0586b7ba16 admin admin] PATH_INFO: `/v3/projects/9988c8d1f7f44ca59b4444e653049885/users/0d0905e558fb482da41c7bf0777c4b58/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:16.498026 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6a3e4-392f-4f93-b3bb-1a0586b7ba16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9988c8d1f7f44ca59b4444e653049885, user_id=0d0905e558fb482da41c7bf0777c4b58, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:16.500751 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6a3e4-392f-4f93-b3bb-1a0586b7ba16 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:16.611776 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1416/2837] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:16 2022] PUT /identity/v3/projects/9988c8d1f7f44ca59b4444e653049885/users/0d0905e558fb482da41c7bf0777c4b58/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:16.635263 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43ee55-c0a0-44de-8f63-8a243d13c959 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:16.635553 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43ee55-c0a0-44de-8f63-8a243d13c959 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:16.635830 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43ee55-c0a0-44de-8f63-8a243d13c959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:16.665122 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-bb43ee55-c0a0-44de-8f63-8a243d13c959 None None] MFA Rules not processed for user `0d0905e558fb482da41c7bf0777c4b58`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:16.666334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bb43ee55-c0a0-44de-8f63-8a243d13c959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:16.733088 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1422/2838] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:16.737810 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6a3e4-392f-4f93-b3bb-1a0586b7ba16 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:16.799685 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6a3e4-392f-4f93-b3bb-1a0586b7ba16 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:16.804384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d06235-da8a-4495-b6f4-8c312008f209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:16.804983 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d06235-da8a-4495-b6f4-8c312008f209 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:16.805262 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d06235-da8a-4495-b6f4-8c312008f209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:16.805534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d06235-da8a-4495-b6f4-8c312008f209 admin admin] PATH_INFO: `/v3/projects` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:16.806154 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d06235-da8a-4495-b6f4-8c312008f209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:16.808045 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d06235-da8a-4495-b6f4-8c312008f209 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:16.937696 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1417/2839] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:16 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:16.941781 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43ee55-c0a0-44de-8f63-8a243d13c959 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:16.953090 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43ee55-c0a0-44de-8f63-8a243d13c959 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:17.012338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7440d2-54c6-4b03-94a4-9b056b39bf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:17.012906 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7440d2-54c6-4b03-94a4-9b056b39bf9e admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:17.013211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7440d2-54c6-4b03-94a4-9b056b39bf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:17.014111 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7440d2-54c6-4b03-94a4-9b056b39bf9e admin admin] PATH_INFO: `/v3/users` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:17.014844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7440d2-54c6-4b03-94a4-9b056b39bf9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:17.016992 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7440d2-54c6-4b03-94a4-9b056b39bf9e admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:17.151992 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1423/2840] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:16 2022] POST /identity/v3/users => generated 299 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:17.157147 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d06235-da8a-4495-b6f4-8c312008f209 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:17.167777 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d06235-da8a-4495-b6f4-8c312008f209 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:17.173382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1db82-2dc0-4303-b227-433b4e61bf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:17.174078 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1db82-2dc0-4303-b227-433b4e61bf94 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:17.174389 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1db82-2dc0-4303-b227-433b4e61bf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:17.174715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1db82-2dc0-4303-b227-433b4e61bf94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:17.175417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1db82-2dc0-4303-b227-433b4e61bf94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:17.177143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1db82-2dc0-4303-b227-433b4e61bf94 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:17.188575 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1418/2841] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:17.192926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7440d2-54c6-4b03-94a4-9b056b39bf9e admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:17.202834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7440d2-54c6-4b03-94a4-9b056b39bf9e admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:17.207334 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e6461-656d-41c0-9c94-55bfeb8f96f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:17.208032 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e6461-656d-41c0-9c94-55bfeb8f96f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:17.208315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e6461-656d-41c0-9c94-55bfeb8f96f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:17.208665 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e6461-656d-41c0-9c94-55bfeb8f96f0 admin admin] PATH_INFO: `/v3/projects/a630c8c22d504c44a9dff9d6d939cc00/users/0bdf5ada68014f00a568adba7daf5d90/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:17.225621 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e6461-656d-41c0-9c94-55bfeb8f96f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a630c8c22d504c44a9dff9d6d939cc00, user_id=0bdf5ada68014f00a568adba7daf5d90, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:17.228292 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e6461-656d-41c0-9c94-55bfeb8f96f0 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:17.340440 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1424/2842] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:17 2022] PUT /identity/v3/projects/a630c8c22d504c44a9dff9d6d939cc00/users/0bdf5ada68014f00a568adba7daf5d90/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:17.345059 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1db82-2dc0-4303-b227-433b4e61bf94 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:17.354994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1db82-2dc0-4303-b227-433b4e61bf94 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:17.359328 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:17.413068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1849c-f7a3-4e21-8c3d-6d8183ee58de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:17.413717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1849c-f7a3-4e21-8c3d-6d8183ee58de admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:17.413998 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1849c-f7a3-4e21-8c3d-6d8183ee58de admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:17.414274 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1849c-f7a3-4e21-8c3d-6d8183ee58de admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:17.414908 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1849c-f7a3-4e21-8c3d-6d8183ee58de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:17.416541 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1849c-f7a3-4e21-8c3d-6d8183ee58de admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:17.426802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1419/2843] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:17.432427 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e6461-656d-41c0-9c94-55bfeb8f96f0 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:17.441329 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e6461-656d-41c0-9c94-55bfeb8f96f0 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:17.446146 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85db6b-6d9c-46a1-bbe4-8caff31ea8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:17.446829 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85db6b-6d9c-46a1-bbe4-8caff31ea8cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:17.447108 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85db6b-6d9c-46a1-bbe4-8caff31ea8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:17.447379 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85db6b-6d9c-46a1-bbe4-8caff31ea8cf admin admin] PATH_INFO: `/v3/projects/a630c8c22d504c44a9dff9d6d939cc00/users/0bdf5ada68014f00a568adba7daf5d90/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:17.449280 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85db6b-6d9c-46a1-bbe4-8caff31ea8cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a630c8c22d504c44a9dff9d6d939cc00, user_id=0bdf5ada68014f00a568adba7daf5d90, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:17.452492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85db6b-6d9c-46a1-bbe4-8caff31ea8cf admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:17.455677 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:17.463232 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1425/2844] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:17 2022] PUT /identity/v3/projects/a630c8c22d504c44a9dff9d6d939cc00/users/0bdf5ada68014f00a568adba7daf5d90/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:17.474673 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d478a52-f8cb-486f-a151-eab8a64e9d4b None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:17.474960 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d478a52-f8cb-486f-a151-eab8a64e9d4b None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:17.475239 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d478a52-f8cb-486f-a151-eab8a64e9d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:17.503222 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-5d478a52-f8cb-486f-a151-eab8a64e9d4b None None] MFA Rules not processed for user `0bdf5ada68014f00a568adba7daf5d90`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:17.504518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5d478a52-f8cb-486f-a151-eab8a64e9d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:17.571417 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1420/2845] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:17 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:17.585285 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85db6b-6d9c-46a1-bbe4-8caff31ea8cf admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:17.587036 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5b85db6b-6d9c-46a1-bbe4-8caff31ea8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:17.647256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85db6b-6d9c-46a1-bbe4-8caff31ea8cf admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:17.651266 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d5112-e607-45ab-8ce7-eda266ea89ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:17.651914 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d5112-e607-45ab-8ce7-eda266ea89ad service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:17.652219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d5112-e607-45ab-8ce7-eda266ea89ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:17.652508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d5112-e607-45ab-8ce7-eda266ea89ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:17.714783 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d5112-e607-45ab-8ce7-eda266ea89ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:17.716456 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d5112-e607-45ab-8ce7-eda266ea89ad service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:17.727139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1426/2846] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:19.877302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d478a52-f8cb-486f-a151-eab8a64e9d4b None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:19.878709 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-5d478a52-f8cb-486f-a151-eab8a64e9d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:19.892559 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d478a52-f8cb-486f-a151-eab8a64e9d4b None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:19.896125 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05df6b4-90b1-4096-adc3-d801812cbc8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:19.896743 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05df6b4-90b1-4096-adc3-d801812cbc8f service cinder] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:19.897018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05df6b4-90b1-4096-adc3-d801812cbc8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:19.897290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05df6b4-90b1-4096-adc3-d801812cbc8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:19.959208 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05df6b4-90b1-4096-adc3-d801812cbc8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:19.961008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05df6b4-90b1-4096-adc3-d801812cbc8f service cinder] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:19.971717 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1421/2847] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:21.828267 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d5112-e607-45ab-8ce7-eda266ea89ad service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:21.846270 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d5112-e607-45ab-8ce7-eda266ea89ad service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:21.850992 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eaa478-bb91-4092-9972-dc7396968268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:21.851608 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eaa478-bb91-4092-9972-dc7396968268 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:21.851889 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eaa478-bb91-4092-9972-dc7396968268 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:21.852186 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eaa478-bb91-4092-9972-dc7396968268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:21.861814 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eaa478-bb91-4092-9972-dc7396968268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:21.863240 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eaa478-bb91-4092-9972-dc7396968268 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:21.875790 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1427/2848] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:26.771525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05df6b4-90b1-4096-adc3-d801812cbc8f service cinder] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:26.806134 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:26.843830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05df6b4-90b1-4096-adc3-d801812cbc8f service cinder] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:26.851240 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:26.852302 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:26.852790 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:26.853279 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] PATH_INFO: `/v3/users/8a1ffec6f7254fcc9e750a50f2e650a6` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:26.855218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a1ffec6f7254fcc9e750a50f2e650a6)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:26.858915 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:27.100000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a1ffec6f7254fcc9e750a50f2e650a6'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:27.100709 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a1ffec6f7254fcc9e750a50f2e650a6'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:27.108085 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a1ffec6f7254fcc9e750a50f2e650a6'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:27.210489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a1ffec6f7254fcc9e750a50f2e650a6'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:27.211195 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a1ffec6f7254fcc9e750a50f2e650a6'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:27.231869 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1422/2849] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:26 2022] DELETE /identity/v3/users/8a1ffec6f7254fcc9e750a50f2e650a6 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:27.236895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aca48d-3c1f-4d69-86d5-ba81c6fdc13d None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:27.237420 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aca48d-3c1f-4d69-86d5-ba81c6fdc13d None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:27.237525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aca48d-3c1f-4d69-86d5-ba81c6fdc13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:27.259895 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-22aca48d-3c1f-4d69-86d5-ba81c6fdc13d None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:27.261150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-22aca48d-3c1f-4d69-86d5-ba81c6fdc13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:27.319073 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1428/2850] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:27.329656 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:27.331083 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:27.387926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298e89b-b7e2-4e2f-9a4c-f733edc2f7df admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:27.391990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b6718-75d8-497e-a655-54286ddf3f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:27.392616 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b6718-75d8-497e-a655-54286ddf3f80 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:27.392898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b6718-75d8-497e-a655-54286ddf3f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:27.393172 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b6718-75d8-497e-a655-54286ddf3f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:27.402265 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b6718-75d8-497e-a655-54286ddf3f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:27.403755 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b6718-75d8-497e-a655-54286ddf3f80 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:27.414490 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1423/2851] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:27.861496 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aca48d-3c1f-4d69-86d5-ba81c6fdc13d None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:27.862953 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-22aca48d-3c1f-4d69-86d5-ba81c6fdc13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:27.875353 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aca48d-3c1f-4d69-86d5-ba81c6fdc13d None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:27.880686 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:27.881417 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:27.881708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:27.882756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] PATH_INFO: `/v3/projects/242e747402e04bfdad34c63223d806a5` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:27.884078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=242e747402e04bfdad34c63223d806a5)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:27.886054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:28.050708 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:28.063738 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242e747402e04bfdad34c63223d806a5', 'request_id': 'req-fb008c19-5786-4240-959e-48fa230ded35'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:28.070135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242e747402e04bfdad34c63223d806a5', 'request_id': 'req-fb008c19-5786-4240-959e-48fa230ded35'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:28.075541 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242e747402e04bfdad34c63223d806a5', 'request_id': 'req-fb008c19-5786-4240-959e-48fa230ded35'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:28.243025 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] The token cache is being invalidate because project 242e747402e04bfdad34c63223d806a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:28.244018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:28.244244 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:28.245643 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1429/2852] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:27 2022] DELETE /identity/v3/projects/242e747402e04bfdad34c63223d806a5 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:28.250866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b6718-75d8-497e-a655-54286ddf3f80 service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:28.261835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b6718-75d8-497e-a655-54286ddf3f80 service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:28.314163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:28.329145 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:28.329825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:28.330105 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:28.330393 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] PATH_INFO: `/v3/users/0d0905e558fb482da41c7bf0777c4b58` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:28.331546 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d0905e558fb482da41c7bf0777c4b58)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:28.335651 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:28.577331 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d0905e558fb482da41c7bf0777c4b58'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:28.577948 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d0905e558fb482da41c7bf0777c4b58'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:28.586286 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d0905e558fb482da41c7bf0777c4b58'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:28.707878 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d0905e558fb482da41c7bf0777c4b58'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:28.734941 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d0905e558fb482da41c7bf0777c4b58'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:28.734941 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1424/2853] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:28 2022] DELETE /identity/v3/users/0d0905e558fb482da41c7bf0777c4b58 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:29.241720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:29.244101 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:29.265572 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb008c19-5786-4240-959e-48fa230ded35 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:29.342525 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:29.343157 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:29.343433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:29.343705 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] PATH_INFO: `/v3/projects/9988c8d1f7f44ca59b4444e653049885` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:29.344891 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9988c8d1f7f44ca59b4444e653049885)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:29.346821 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:29.549399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9988c8d1f7f44ca59b4444e653049885', 'request_id': 'req-c9c66cfe-317b-4815-94a0-c10414524d2b'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:29.555666 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9988c8d1f7f44ca59b4444e653049885', 'request_id': 'req-c9c66cfe-317b-4815-94a0-c10414524d2b'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:29.561651 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9988c8d1f7f44ca59b4444e653049885', 'request_id': 'req-c9c66cfe-317b-4815-94a0-c10414524d2b'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:29.647555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] The token cache is being invalidate because project 9988c8d1f7f44ca59b4444e653049885 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:29.651200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:29.651200 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:29.651619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1430/2854] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:29 2022] DELETE /identity/v3/projects/9988c8d1f7f44ca59b4444e653049885 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:29.657112 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:29.658898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:29.672455 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd102b-af7e-4d41-bc72-3295ca7fc2bd admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:29.730297 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfdcdfe-7594-4920-8a80-a16ad53ae2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:29.730945 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfdcdfe-7594-4920-8a80-a16ad53ae2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:29.731209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfdcdfe-7594-4920-8a80-a16ad53ae2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:29.731483 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfdcdfe-7594-4920-8a80-a16ad53ae2cb admin admin] PATH_INFO: `/v3/users/0bdf5ada68014f00a568adba7daf5d90` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:29.732642 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfdcdfe-7594-4920-8a80-a16ad53ae2cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bdf5ada68014f00a568adba7daf5d90)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:29.734560 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfdcdfe-7594-4920-8a80-a16ad53ae2cb admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:29.966732 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8dfdcdfe-7594-4920-8a80-a16ad53ae2cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bdf5ada68014f00a568adba7daf5d90'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:29.967772 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8dfdcdfe-7594-4920-8a80-a16ad53ae2cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bdf5ada68014f00a568adba7daf5d90'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:29.980736 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8dfdcdfe-7594-4920-8a80-a16ad53ae2cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bdf5ada68014f00a568adba7daf5d90'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.050276 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8dfdcdfe-7594-4920-8a80-a16ad53ae2cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bdf5ada68014f00a568adba7daf5d90'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.050984 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-8dfdcdfe-7594-4920-8a80-a16ad53ae2cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bdf5ada68014f00a568adba7daf5d90'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.057971 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:30.074475 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1425/2855] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:29 2022] DELETE /identity/v3/users/0bdf5ada68014f00a568adba7daf5d90 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:30.593556 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:30.595237 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:30.611775 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c66cfe-317b-4815-94a0-c10414524d2b admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:30.674173 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:30.675081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413a0b7-2303-41ba-9473-8dbb50ff0d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:30.675818 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413a0b7-2303-41ba-9473-8dbb50ff0d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:30.676093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413a0b7-2303-41ba-9473-8dbb50ff0d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:30.676403 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413a0b7-2303-41ba-9473-8dbb50ff0d20 admin admin] PATH_INFO: `/v3/projects/a630c8c22d504c44a9dff9d6d939cc00` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:30.677791 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413a0b7-2303-41ba-9473-8dbb50ff0d20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a630c8c22d504c44a9dff9d6d939cc00)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:30.679735 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413a0b7-2303-41ba-9473-8dbb50ff0d20 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:30.814399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f413a0b7-2303-41ba-9473-8dbb50ff0d20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a630c8c22d504c44a9dff9d6d939cc00', 'request_id': 'req-f413a0b7-2303-41ba-9473-8dbb50ff0d20'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.820487 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f413a0b7-2303-41ba-9473-8dbb50ff0d20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a630c8c22d504c44a9dff9d6d939cc00', 'request_id': 'req-f413a0b7-2303-41ba-9473-8dbb50ff0d20'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.825751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f413a0b7-2303-41ba-9473-8dbb50ff0d20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a630c8c22d504c44a9dff9d6d939cc00', 'request_id': 'req-f413a0b7-2303-41ba-9473-8dbb50ff0d20'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.952816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f413a0b7-2303-41ba-9473-8dbb50ff0d20 admin admin] The token cache is being invalidate because project a630c8c22d504c44a9dff9d6d939cc00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:30.953107 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f413a0b7-2303-41ba-9473-8dbb50ff0d20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.953724 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-f413a0b7-2303-41ba-9473-8dbb50ff0d20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.955242 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1431/2856] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:30 2022] DELETE /identity/v3/projects/a630c8c22d504c44a9dff9d6d939cc00 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:30.960401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbe3ac-55f1-4cde-abe3-70ecc78082a1 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:30.960898 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbe3ac-55f1-4cde-abe3-70ecc78082a1 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:30.961016 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbe3ac-55f1-4cde-abe3-70ecc78082a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:30.980420 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-57dbe3ac-55f1-4cde-abe3-70ecc78082a1 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:30.981613 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-57dbe3ac-55f1-4cde-abe3-70ecc78082a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:31.040814 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1426/2857] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:31.049989 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148566d-916e-4360-8757-518913dc6760 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.050276 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148566d-916e-4360-8757-518913dc6760 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.050549 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148566d-916e-4360-8757-518913dc6760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.070313 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-8148566d-916e-4360-8757-518913dc6760 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:31.071518 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-8148566d-916e-4360-8757-518913dc6760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:31.080147 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1432/2858] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:31.084434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbe3ac-55f1-4cde-abe3-70ecc78082a1 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:31.094799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbe3ac-55f1-4cde-abe3-70ecc78082a1 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:31.099461 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54be48-c7db-43ad-8572-9730e74484c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:31.100068 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54be48-c7db-43ad-8572-9730e74484c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.100384 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54be48-c7db-43ad-8572-9730e74484c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.100656 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54be48-c7db-43ad-8572-9730e74484c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.101330 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54be48-c7db-43ad-8572-9730e74484c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:39:31.101628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54be48-c7db-43ad-8572-9730e74484c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:31.103205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54be48-c7db-43ad-8572-9730e74484c2 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:31.114957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1427/2859] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:39:31.119321 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148566d-916e-4360-8757-518913dc6760 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:31.128720 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148566d-916e-4360-8757-518913dc6760 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:31.133359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6efe6-471f-4e8c-9c47-d2b4b4016b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:31.133955 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6efe6-471f-4e8c-9c47-d2b4b4016b33 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.134235 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6efe6-471f-4e8c-9c47-d2b4b4016b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.134508 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6efe6-471f-4e8c-9c47-d2b4b4016b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.135144 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6efe6-471f-4e8c-9c47-d2b4b4016b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:31.137078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6efe6-471f-4e8c-9c47-d2b4b4016b33 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:31.244880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1433/2860] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:31 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:31.249339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54be48-c7db-43ad-8572-9730e74484c2 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:31.259534 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54be48-c7db-43ad-8572-9730e74484c2 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:31.318401 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb71a57-1d00-472b-9b0e-736e4246fbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:31.319013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb71a57-1d00-472b-9b0e-736e4246fbc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.319292 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb71a57-1d00-472b-9b0e-736e4246fbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.319565 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb71a57-1d00-472b-9b0e-736e4246fbc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.320234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb71a57-1d00-472b-9b0e-736e4246fbc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:31.322220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb71a57-1d00-472b-9b0e-736e4246fbc9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:31.445243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1428/2861] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:31 2022] POST /identity/v3/users => generated 283 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:31.449384 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6efe6-471f-4e8c-9c47-d2b4b4016b33 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:31.460277 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6efe6-471f-4e8c-9c47-d2b4b4016b33 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:31.464938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11675979-52b6-4187-8857-48216df95ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:31.465538 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11675979-52b6-4187-8857-48216df95ead admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.465847 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11675979-52b6-4187-8857-48216df95ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.466095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11675979-52b6-4187-8857-48216df95ead admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.466808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11675979-52b6-4187-8857-48216df95ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:31.468470 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11675979-52b6-4187-8857-48216df95ead admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:31.478696 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1434/2862] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:31.482848 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb71a57-1d00-472b-9b0e-736e4246fbc9 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:31.493582 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb71a57-1d00-472b-9b0e-736e4246fbc9 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:31.499177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484d350-ba94-4916-a731-53b7e6929e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:31.500100 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484d350-ba94-4916-a731-53b7e6929e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.500545 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484d350-ba94-4916-a731-53b7e6929e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.501032 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484d350-ba94-4916-a731-53b7e6929e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.501841 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484d350-ba94-4916-a731-53b7e6929e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:31.503721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484d350-ba94-4916-a731-53b7e6929e27 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:31.516018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1429/2863] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:31.520337 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11675979-52b6-4187-8857-48216df95ead admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:31.531049 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11675979-52b6-4187-8857-48216df95ead admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:31.535718 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93d708-a303-4009-af7c-3963aff0cf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:31.536405 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93d708-a303-4009-af7c-3963aff0cf5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.536688 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93d708-a303-4009-af7c-3963aff0cf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.536962 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93d708-a303-4009-af7c-3963aff0cf5a admin admin] PATH_INFO: `/v3/projects/aa83510d081441ecaa6c442496905086/users/ce43faeadc56401cbc532177e0e7e726/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.551521 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93d708-a303-4009-af7c-3963aff0cf5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa83510d081441ecaa6c442496905086, user_id=ce43faeadc56401cbc532177e0e7e726, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:31.554201 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93d708-a303-4009-af7c-3963aff0cf5a admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:31.657095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1435/2864] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:31 2022] PUT /identity/v3/projects/aa83510d081441ecaa6c442496905086/users/ce43faeadc56401cbc532177e0e7e726/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:31.667362 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68611d-cf91-4bac-9131-c36f8e382706 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.667649 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68611d-cf91-4bac-9131-c36f8e382706 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.667928 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68611d-cf91-4bac-9131-c36f8e382706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.695218 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-db68611d-cf91-4bac-9131-c36f8e382706 None None] MFA Rules not processed for user `ce43faeadc56401cbc532177e0e7e726`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:31.696486 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-db68611d-cf91-4bac-9131-c36f8e382706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:31.737761 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:31.759980 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1430/2865] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:39:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:31.771782 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93d708-a303-4009-af7c-3963aff0cf5a admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:31.773758 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7f93d708-a303-4009-af7c-3963aff0cf5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:31.830211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93d708-a303-4009-af7c-3963aff0cf5a admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:31.834113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7f08c-22e6-406c-b648-9b24d6785863 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:31.839490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7f08c-22e6-406c-b648-9b24d6785863 service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.839490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7f08c-22e6-406c-b648-9b24d6785863 service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.839490 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7f08c-22e6-406c-b648-9b24d6785863 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.844933 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7f08c-22e6-406c-b648-9b24d6785863 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:31.846399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7f08c-22e6-406c-b648-9b24d6785863 service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:31.855099 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1436/2866] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:41.851288 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:41.861353 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=8 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:13.007770 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68611d-cf91-4bac-9131-c36f8e382706 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:13.070290 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68611d-cf91-4bac-9131-c36f8e382706 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:13.074911 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:13.077312 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:13.077312 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:13.077312 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] PATH_INFO: `/v3/users/ce43faeadc56401cbc532177e0e7e726` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:13.077312 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce43faeadc56401cbc532177e0e7e726)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:13.079092 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:13.286012 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce43faeadc56401cbc532177e0e7e726'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:13.286606 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce43faeadc56401cbc532177e0e7e726'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:13.293542 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce43faeadc56401cbc532177e0e7e726'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:13.396007 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce43faeadc56401cbc532177e0e7e726'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:13.397166 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce43faeadc56401cbc532177e0e7e726'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:13.415702 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1431/2867] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:40:13 2022] DELETE /identity/v3/users/ce43faeadc56401cbc532177e0e7e726 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:13.420493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a40215-b9b0-4a1f-9f6d-0acae3aa2503 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:13.421001 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a40215-b9b0-4a1f-9f6d-0acae3aa2503 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:13.421116 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a40215-b9b0-4a1f-9f6d-0acae3aa2503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:13.439550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-a8a40215-b9b0-4a1f-9f6d-0acae3aa2503 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:13.440766 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a8a40215-b9b0-4a1f-9f6d-0acae3aa2503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:13.500023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1437/2868] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:13.509873 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:13.511265 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:13.568577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1e9d7-75db-4db3-b44e-e7edb300cf27 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:13.572101 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66553cd0-add0-4e44-8a01-b7c7113a0de0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:13.572728 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66553cd0-add0-4e44-8a01-b7c7113a0de0 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:13.573009 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66553cd0-add0-4e44-8a01-b7c7113a0de0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:13.573285 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66553cd0-add0-4e44-8a01-b7c7113a0de0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:13.582216 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66553cd0-add0-4e44-8a01-b7c7113a0de0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:13.583646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66553cd0-add0-4e44-8a01-b7c7113a0de0 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:13.593416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1432/2869] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:40:14.074367 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a40215-b9b0-4a1f-9f6d-0acae3aa2503 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:14.076062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a8a40215-b9b0-4a1f-9f6d-0acae3aa2503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:14.089138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a40215-b9b0-4a1f-9f6d-0acae3aa2503 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:14.093909 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:14.094690 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:14.094980 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:14.095314 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07 admin admin] PATH_INFO: `/v3/projects/aa83510d081441ecaa6c442496905086` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:14.096479 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa83510d081441ecaa6c442496905086)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:14.098454 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:14.238682 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa83510d081441ecaa6c442496905086', 'request_id': 'req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:14.245021 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa83510d081441ecaa6c442496905086', 'request_id': 'req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:14.250585 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa83510d081441ecaa6c442496905086', 'request_id': 'req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:14.339953 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07 admin admin] The token cache is being invalidate because project aa83510d081441ecaa6c442496905086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:40:14.340182 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:14.340808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-e9fd1fa1-f560-4af1-85f1-ea7c0d118a07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:14.342302 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1438/2870] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:40:14 2022] DELETE /identity/v3/projects/aa83510d081441ecaa6c442496905086 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:14.347167 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab022da-ab5a-4521-be5d-bdbc5f3e4665 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:14.347654 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab022da-ab5a-4521-be5d-bdbc5f3e4665 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:14.347753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab022da-ab5a-4521-be5d-bdbc5f3e4665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:14.367499 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-cab022da-ab5a-4521-be5d-bdbc5f3e4665 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:14.373249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-cab022da-ab5a-4521-be5d-bdbc5f3e4665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:14.386132 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:14.428374 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1433/2871] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:14.437330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd9862-e196-46b2-9898-9c32bf3e3c3b None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:14.437834 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd9862-e196-46b2-9898-9c32bf3e3c3b None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:14.437950 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd9862-e196-46b2-9898-9c32bf3e3c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:14.458492 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-5fcd9862-e196-46b2-9898-9c32bf3e3c3b None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:14.459866 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-5fcd9862-e196-46b2-9898-9c32bf3e3c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:14.467787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1439/2872] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:14.472247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab022da-ab5a-4521-be5d-bdbc5f3e4665 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:14.482216 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab022da-ab5a-4521-be5d-bdbc5f3e4665 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:14.487364 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be0dca-9931-4526-98a9-b307baaaf297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:14.487959 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be0dca-9931-4526-98a9-b307baaaf297 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:14.488283 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be0dca-9931-4526-98a9-b307baaaf297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:14.488563 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be0dca-9931-4526-98a9-b307baaaf297 admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:14.489260 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be0dca-9931-4526-98a9-b307baaaf297 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:40:14.489561 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be0dca-9931-4526-98a9-b307baaaf297 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:14.491662 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be0dca-9931-4526-98a9-b307baaaf297 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:14.503185 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1434/2873] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:40:14.508279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd9862-e196-46b2-9898-9c32bf3e3c3b None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:14.517516 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd9862-e196-46b2-9898-9c32bf3e3c3b None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:14.522286 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774977ef-30fc-4466-a133-ad2f958a9970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:14.522884 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774977ef-30fc-4466-a133-ad2f958a9970 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:14.523168 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774977ef-30fc-4466-a133-ad2f958a9970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:14.523442 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774977ef-30fc-4466-a133-ad2f958a9970 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:14.524377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774977ef-30fc-4466-a133-ad2f958a9970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:14.526315 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774977ef-30fc-4466-a133-ad2f958a9970 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:14.622136 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1440/2874] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:40:14 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:40:14.623366 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be0dca-9931-4526-98a9-b307baaaf297 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:14.633990 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be0dca-9931-4526-98a9-b307baaaf297 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:14.691866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fd721-9ac2-4f30-afa9-6104a28d2e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:14.692509 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fd721-9ac2-4f30-afa9-6104a28d2e38 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:14.692786 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fd721-9ac2-4f30-afa9-6104a28d2e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:14.693058 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fd721-9ac2-4f30-afa9-6104a28d2e38 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:14.693733 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fd721-9ac2-4f30-afa9-6104a28d2e38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:14.695622 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fd721-9ac2-4f30-afa9-6104a28d2e38 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:14.808155 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1435/2875] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:40:14 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:40:14.812320 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774977ef-30fc-4466-a133-ad2f958a9970 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:14.822595 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774977ef-30fc-4466-a133-ad2f958a9970 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:14.827176 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2e152-4498-4912-9e17-0ebfdf4a61fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:14.827774 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2e152-4498-4912-9e17-0ebfdf4a61fb admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:14.828055 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2e152-4498-4912-9e17-0ebfdf4a61fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:14.828359 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2e152-4498-4912-9e17-0ebfdf4a61fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:14.829045 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2e152-4498-4912-9e17-0ebfdf4a61fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:14.830632 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2e152-4498-4912-9e17-0ebfdf4a61fb admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:14.841026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1441/2876] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:14.845055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fd721-9ac2-4f30-afa9-6104a28d2e38 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:14.855628 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fd721-9ac2-4f30-afa9-6104a28d2e38 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:14.860309 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2687b5-4662-441a-9029-6af7ab06bd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:14.860902 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2687b5-4662-441a-9029-6af7ab06bd9d admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:14.861186 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2687b5-4662-441a-9029-6af7ab06bd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:14.862235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2687b5-4662-441a-9029-6af7ab06bd9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:14.863243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2687b5-4662-441a-9029-6af7ab06bd9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:14.864887 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2687b5-4662-441a-9029-6af7ab06bd9d admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:14.877342 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1436/2877] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:14.881373 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2e152-4498-4912-9e17-0ebfdf4a61fb admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:14.891113 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2e152-4498-4912-9e17-0ebfdf4a61fb admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:14.895750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf1997-8ed1-45a1-8a3b-ad0d8bce12d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:14.896966 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf1997-8ed1-45a1-8a3b-ad0d8bce12d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:14.897248 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf1997-8ed1-45a1-8a3b-ad0d8bce12d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:14.897520 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf1997-8ed1-45a1-8a3b-ad0d8bce12d2 admin admin] PATH_INFO: `/v3/projects/9bab8a45db344c0ea68c26a38b126809/users/1a9c77cb2e0a4aa19c3cc289b9ffb5e4/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:14.912815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf1997-8ed1-45a1-8a3b-ad0d8bce12d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bab8a45db344c0ea68c26a38b126809, user_id=1a9c77cb2e0a4aa19c3cc289b9ffb5e4, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:14.915438 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf1997-8ed1-45a1-8a3b-ad0d8bce12d2 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:15.021346 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1442/2878] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:40:14 2022] PUT /identity/v3/projects/9bab8a45db344c0ea68c26a38b126809/users/1a9c77cb2e0a4aa19c3cc289b9ffb5e4/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:15.031988 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80110217-58e7-4d4b-ad03-64f4e9ea92d6 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:15.032315 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80110217-58e7-4d4b-ad03-64f4e9ea92d6 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:15.032593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80110217-58e7-4d4b-ad03-64f4e9ea92d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:15.061080 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-80110217-58e7-4d4b-ad03-64f4e9ea92d6 None None] MFA Rules not processed for user `1a9c77cb2e0a4aa19c3cc289b9ffb5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:15.062268 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-80110217-58e7-4d4b-ad03-64f4e9ea92d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:15.126433 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1437/2879] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:40:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:15.146178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf1997-8ed1-45a1-8a3b-ad0d8bce12d2 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:15.148901 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e8bf1997-8ed1-45a1-8a3b-ad0d8bce12d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:15.170377 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:15.206520 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf1997-8ed1-45a1-8a3b-ad0d8bce12d2 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:15.210859 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda83e16-4fa4-4b49-8deb-0178eec823ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d90a7ed44fd4a1fa94b9d44939df425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:15.211455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda83e16-4fa4-4b49-8deb-0178eec823ea service cinder] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:15.211827 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda83e16-4fa4-4b49-8deb-0178eec823ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:15.212105 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda83e16-4fa4-4b49-8deb-0178eec823ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:15.220565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda83e16-4fa4-4b49-8deb-0178eec823ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:15.222011 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda83e16-4fa4-4b49-8deb-0178eec823ea service cinder] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:15.231330 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1443/2880] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:40:17.169241 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec643b-7c0e-4c5a-a79f-2651562c7455 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:17.169753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec643b-7c0e-4c5a-a79f-2651562c7455 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:17.169856 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec643b-7c0e-4c5a-a79f-2651562c7455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:17.188326 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-3fec643b-7c0e-4c5a-a79f-2651562c7455 None None] MFA Rules not processed for user `1a9c77cb2e0a4aa19c3cc289b9ffb5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:17.189518 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3fec643b-7c0e-4c5a-a79f-2651562c7455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:17.196348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1438/2881] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:40:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:17.816967 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda83e16-4fa4-4b49-8deb-0178eec823ea service cinder] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:17.819405 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bda83e16-4fa4-4b49-8deb-0178eec823ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:17.894089 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda83e16-4fa4-4b49-8deb-0178eec823ea service cinder] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:17.899489 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f897bb-791b-413a-8c96-1c2e7bf3149b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:17.900095 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f897bb-791b-413a-8c96-1c2e7bf3149b service placement] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:17.900413 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f897bb-791b-413a-8c96-1c2e7bf3149b service placement] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:17.900684 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f897bb-791b-413a-8c96-1c2e7bf3149b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:17.902411 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-f8f897bb-791b-413a-8c96-1c2e7bf3149b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:17.914018 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f897bb-791b-413a-8c96-1c2e7bf3149b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:17.915676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f897bb-791b-413a-8c96-1c2e7bf3149b service placement] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:17.925828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1444/2882] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:40:27.182904 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:27.921481 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=49 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:28.393593 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec643b-7c0e-4c5a-a79f-2651562c7455 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:28.395211 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3fec643b-7c0e-4c5a-a79f-2651562c7455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:28.468626 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec643b-7c0e-4c5a-a79f-2651562c7455 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:28.474228 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc925c2-72af-476b-ae49-7d5bb0d0db60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:28.474830 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc925c2-72af-476b-ae49-7d5bb0d0db60 service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:28.475113 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc925c2-72af-476b-ae49-7d5bb0d0db60 service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:28.475376 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc925c2-72af-476b-ae49-7d5bb0d0db60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:28.477159 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-9cc925c2-72af-476b-ae49-7d5bb0d0db60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:28.489448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc925c2-72af-476b-ae49-7d5bb0d0db60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:28.491546 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc925c2-72af-476b-ae49-7d5bb0d0db60 service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:28.509156 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1439/2883] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:40:34.741053 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f897bb-791b-413a-8c96-1c2e7bf3149b service placement] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:34.751630 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f897bb-791b-413a-8c96-1c2e7bf3149b service placement] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:34.757335 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eeb45f-9679-4d3b-8d94-135df6f9a4f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:34.757950 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eeb45f-9679-4d3b-8d94-135df6f9a4f9 service placement] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:34.758225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eeb45f-9679-4d3b-8d94-135df6f9a4f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:34.758496 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eeb45f-9679-4d3b-8d94-135df6f9a4f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:34.760256 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a4eeb45f-9679-4d3b-8d94-135df6f9a4f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:34.771555 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eeb45f-9679-4d3b-8d94-135df6f9a4f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:34.773151 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eeb45f-9679-4d3b-8d94-135df6f9a4f9 service placement] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:34.783481 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1445/2884] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:40:35.907941 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992d398-eb50-4807-bc7b-5bbb96f0f45e None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:35.908434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992d398-eb50-4807-bc7b-5bbb96f0f45e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:35.908677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992d398-eb50-4807-bc7b-5bbb96f0f45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:35.927686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-1992d398-eb50-4807-bc7b-5bbb96f0f45e None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:35.929448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1992d398-eb50-4807-bc7b-5bbb96f0f45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:35.991721 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1440/2885] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:40:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:36.122692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eeb45f-9679-4d3b-8d94-135df6f9a4f9 service placement] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:36.122692 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a4eeb45f-9679-4d3b-8d94-135df6f9a4f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:36.213515 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eeb45f-9679-4d3b-8d94-135df6f9a4f9 service placement] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:36.218530 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11b16a-8adb-4258-97ae-a4c735f60fa7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:36.219124 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11b16a-8adb-4258-97ae-a4c735f60fa7 service swift] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:36.219404 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11b16a-8adb-4258-97ae-a4c735f60fa7 service swift] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:36.219683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11b16a-8adb-4258-97ae-a4c735f60fa7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:36.228338 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df11b16a-8adb-4258-97ae-a4c735f60fa7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:36.229915 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df11b16a-8adb-4258-97ae-a4c735f60fa7 service swift] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:36.237816 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1446/2886] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:40:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:40:37.317294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4894d-e3d0-4253-b327-2ec56afc1932 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:37.317801 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4894d-e3d0-4253-b327-2ec56afc1932 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:37.317925 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4894d-e3d0-4253-b327-2ec56afc1932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:37.338163 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-71e4894d-e3d0-4253-b327-2ec56afc1932 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:37.339372 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-71e4894d-e3d0-4253-b327-2ec56afc1932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:37.346574 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1441/2887] 10.222.0.85 () {58 vars in 967 bytes} [Mon Aug 29 20:40:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:37.358494 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11b16a-8adb-4258-97ae-a4c735f60fa7 service swift] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:37.369755 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11b16a-8adb-4258-97ae-a4c735f60fa7 service swift] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:37.374821 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d5d08-be5d-4e43-b305-853f66f3698f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:37.375641 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d5d08-be5d-4e43-b305-853f66f3698f service swift] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:37.375924 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d5d08-be5d-4e43-b305-853f66f3698f service swift] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:37.376228 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d5d08-be5d-4e43-b305-853f66f3698f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:37.385082 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d5d08-be5d-4e43-b305-853f66f3698f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:37.386660 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d5d08-be5d-4e43-b305-853f66f3698f service swift] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:37.394808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1447/2888] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:40:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:40:38.108577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c78d8-dbde-49fe-aa56-efc2b0617a44 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:38.114623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c78d8-dbde-49fe-aa56-efc2b0617a44 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:38.114623 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c78d8-dbde-49fe-aa56-efc2b0617a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:38.129235 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-0d7c78d8-dbde-49fe-aa56-efc2b0617a44 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:38.130363 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-0d7c78d8-dbde-49fe-aa56-efc2b0617a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:38.138448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1442/2889] 10.222.0.85 () {58 vars in 1009 bytes} [Mon Aug 29 20:40:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:38.150570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d5d08-be5d-4e43-b305-853f66f3698f service swift] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:38.161213 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d5d08-be5d-4e43-b305-853f66f3698f service swift] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:38.166125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5cec1-e76e-4e35-9030-fe8c9c4506c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:38.166733 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5cec1-e76e-4e35-9030-fe8c9c4506c1 service swift] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:38.167012 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5cec1-e76e-4e35-9030-fe8c9c4506c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:38.167279 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5cec1-e76e-4e35-9030-fe8c9c4506c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:38.175399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5cec1-e76e-4e35-9030-fe8c9c4506c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:38.177066 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5cec1-e76e-4e35-9030-fe8c9c4506c1 service swift] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:38.185465 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1448/2890] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:40:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:40:38.849043 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af5ef5-188b-45a4-8733-e87db6423cc7 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:38.849536 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af5ef5-188b-45a4-8733-e87db6423cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:38.849650 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af5ef5-188b-45a4-8733-e87db6423cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:38.869093 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-23af5ef5-188b-45a4-8733-e87db6423cc7 None None] MFA Rules not processed for user `2d270713965b47fd9dbb2bcbec2634d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:38.870273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-23af5ef5-188b-45a4-8733-e87db6423cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:38.877306 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1443/2891] 10.222.0.85 () {58 vars in 967 bytes} [Mon Aug 29 20:40:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:38.889676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5cec1-e76e-4e35-9030-fe8c9c4506c1 service swift] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:38.901163 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5cec1-e76e-4e35-9030-fe8c9c4506c1 service swift] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:38.906159 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ca0bf-02da-4432-a380-e5d49072cf33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd52f84245a2142e69c309aa709a7d921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:38.906748 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ca0bf-02da-4432-a380-e5d49072cf33 service swift] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:38.907030 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ca0bf-02da-4432-a380-e5d49072cf33 service swift] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:38.907302 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ca0bf-02da-4432-a380-e5d49072cf33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:38.915339 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ca0bf-02da-4432-a380-e5d49072cf33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:38.916945 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ca0bf-02da-4432-a380-e5d49072cf33 service swift] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:38.924821 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1449/2892] 10.222.0.85 () {60 vars in 1384 bytes} [Mon Aug 29 20:40:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:40:41.585635 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af5ef5-188b-45a4-8733-e87db6423cc7 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:41.648835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af5ef5-188b-45a4-8733-e87db6423cc7 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:41.653489 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:41.654116 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:41.654400 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:41.654675 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] PATH_INFO: `/v3/users/1a9c77cb2e0a4aa19c3cc289b9ffb5e4` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:41.655803 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a9c77cb2e0a4aa19c3cc289b9ffb5e4)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:41.657759 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:41.660129 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=100 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:41.896391 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a9c77cb2e0a4aa19c3cc289b9ffb5e4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:41.897066 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a9c77cb2e0a4aa19c3cc289b9ffb5e4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:41.904099 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a9c77cb2e0a4aa19c3cc289b9ffb5e4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:41.997547 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a9c77cb2e0a4aa19c3cc289b9ffb5e4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:41.998177 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a9c77cb2e0a4aa19c3cc289b9ffb5e4'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:42.019318 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1444/2893] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:40:41 2022] DELETE /identity/v3/users/1a9c77cb2e0a4aa19c3cc289b9ffb5e4 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:42.021201 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a5a53-9c93-4782-a8a5-056520fc1419 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:42.021740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a5a53-9c93-4782-a8a5-056520fc1419 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:42.021740 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a5a53-9c93-4782-a8a5-056520fc1419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:42.041837 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e23a5a53-9c93-4782-a8a5-056520fc1419 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:42.043029 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e23a5a53-9c93-4782-a8a5-056520fc1419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:42.100754 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1450/2894] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:42.110477 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:42.111886 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:42.168122 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b338ff5-5796-49eb-85d5-04595559a76f admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:42.171715 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c24cf5-593c-4d62-aecd-4db1fff67a01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:42.172343 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c24cf5-593c-4d62-aecd-4db1fff67a01 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:42.172624 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c24cf5-593c-4d62-aecd-4db1fff67a01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:42.172900 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c24cf5-593c-4d62-aecd-4db1fff67a01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:42.182299 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c24cf5-593c-4d62-aecd-4db1fff67a01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:42.183725 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c24cf5-593c-4d62-aecd-4db1fff67a01 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:42.193018 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1445/2895] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:40:42.451565 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a5a53-9c93-4782-a8a5-056520fc1419 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:42.453062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e23a5a53-9c93-4782-a8a5-056520fc1419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:42.467570 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a5a53-9c93-4782-a8a5-056520fc1419 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:42.472215 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:42.472842 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:42.473245 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:42.473557 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7 admin admin] PATH_INFO: `/v3/projects/9bab8a45db344c0ea68c26a38b126809` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:42.474808 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bab8a45db344c0ea68c26a38b126809)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:42.477391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:42.489500 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=100 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:42.678219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bab8a45db344c0ea68c26a38b126809', 'request_id': 'req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:42.684391 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bab8a45db344c0ea68c26a38b126809', 'request_id': 'req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:42.689713 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bab8a45db344c0ea68c26a38b126809', 'request_id': 'req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:42.809697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7 admin admin] The token cache is being invalidate because project 9bab8a45db344c0ea68c26a38b126809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:40:42.810509 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:42.811178 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-2a63a222-cb9d-418e-bde3-bbb5bd2e51f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:42.814209 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1451/2896] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:40:42 2022] DELETE /identity/v3/projects/9bab8a45db344c0ea68c26a38b126809 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:42.819336 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c1551-1c9e-4ff9-bf44-5e084c663346 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:42.835000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c1551-1c9e-4ff9-bf44-5e084c663346 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:42.835000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c1551-1c9e-4ff9-bf44-5e084c663346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:42.839350 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-f89c1551-1c9e-4ff9-bf44-5e084c663346 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:42.840577 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-f89c1551-1c9e-4ff9-bf44-5e084c663346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:42.898852 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1446/2897] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:42.906429 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dfd5a-811a-40ee-8691-34624d136e43 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:42.906719 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dfd5a-811a-40ee-8691-34624d136e43 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:42.906996 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dfd5a-811a-40ee-8691-34624d136e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:42.927495 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-cf5dfd5a-811a-40ee-8691-34624d136e43 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:42.928787 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-cf5dfd5a-811a-40ee-8691-34624d136e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:42.937448 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1452/2898] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:42.941567 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c1551-1c9e-4ff9-bf44-5e084c663346 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:42.951270 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c1551-1c9e-4ff9-bf44-5e084c663346 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:42.955779 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f78af-117c-4c69-9a6e-2c6a8aa5032c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:42.956476 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f78af-117c-4c69-9a6e-2c6a8aa5032c admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:42.956763 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f78af-117c-4c69-9a6e-2c6a8aa5032c admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:42.957037 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f78af-117c-4c69-9a6e-2c6a8aa5032c admin admin] PATH_INFO: `/v3/domains` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:42.957677 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f78af-117c-4c69-9a6e-2c6a8aa5032c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81859) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:40:42.958051 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f78af-117c-4c69-9a6e-2c6a8aa5032c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:42.960138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f78af-117c-4c69-9a6e-2c6a8aa5032c admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:42.971416 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1447/2899] 10.222.0.85 () {58 vars in 1153 bytes} [Mon Aug 29 20:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:40:42.976242 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5dfd5a-811a-40ee-8691-34624d136e43 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:42.985395 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5dfd5a-811a-40ee-8691-34624d136e43 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:42.990260 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef37e41-613c-47e0-934e-249fbce310c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:42.990844 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef37e41-613c-47e0-934e-249fbce310c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:42.991123 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef37e41-613c-47e0-934e-249fbce310c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:42.991399 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef37e41-613c-47e0-934e-249fbce310c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:42.992023 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef37e41-613c-47e0-934e-249fbce310c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:42.995293 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef37e41-613c-47e0-934e-249fbce310c3 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:43.119019 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1453/2900] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:40:42 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:40:43.123100 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f78af-117c-4c69-9a6e-2c6a8aa5032c admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:43.133686 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f78af-117c-4c69-9a6e-2c6a8aa5032c admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:43.141514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:43.188857 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a317886-8266-43f7-9eab-dc5644477128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:43.189459 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a317886-8266-43f7-9eab-dc5644477128 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:43.189737 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a317886-8266-43f7-9eab-dc5644477128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:43.190008 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a317886-8266-43f7-9eab-dc5644477128 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:43.190666 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a317886-8266-43f7-9eab-dc5644477128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:43.192600 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a317886-8266-43f7-9eab-dc5644477128 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:43.306581 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1448/2901] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:40:43 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:40:43.310644 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef37e41-613c-47e0-934e-249fbce310c3 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:43.321211 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef37e41-613c-47e0-934e-249fbce310c3 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:43.325860 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a85d0-882b-4003-afa8-3acc55a02a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:43.326458 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a85d0-882b-4003-afa8-3acc55a02a6d admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:43.326748 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a85d0-882b-4003-afa8-3acc55a02a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:43.327019 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a85d0-882b-4003-afa8-3acc55a02a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:43.327654 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a85d0-882b-4003-afa8-3acc55a02a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:43.329297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a85d0-882b-4003-afa8-3acc55a02a6d admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:43.339815 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1454/2902] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:43.343708 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a317886-8266-43f7-9eab-dc5644477128 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:43.354249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a317886-8266-43f7-9eab-dc5644477128 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:43.358966 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd25a9-7857-4faf-a0e2-dee2d69ee581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:43.359555 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd25a9-7857-4faf-a0e2-dee2d69ee581 admin admin] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:43.360055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd25a9-7857-4faf-a0e2-dee2d69ee581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:43.360055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd25a9-7857-4faf-a0e2-dee2d69ee581 admin admin] PATH_INFO: `/v3/roles` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:43.360852 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd25a9-7857-4faf-a0e2-dee2d69ee581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:43.362448 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd25a9-7857-4faf-a0e2-dee2d69ee581 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:43.373138 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1449/2903] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:43.400627 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a85d0-882b-4003-afa8-3acc55a02a6d admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:43.413371 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a85d0-882b-4003-afa8-3acc55a02a6d admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:43.418245 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e544869-e8f6-4934-abc3-e7bb40703c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:43.418892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e544869-e8f6-4934-abc3-e7bb40703c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:43.419173 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e544869-e8f6-4934-abc3-e7bb40703c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:43.419455 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e544869-e8f6-4934-abc3-e7bb40703c74 admin admin] PATH_INFO: `/v3/projects/a55662f8acc54509b089b401af78ea9b/users/1d30c32e7ee441a38cdd67660fc81cd2/roles/4f10622fcf4c4f708cd6b652d886ba3e` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:43.434093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e544869-e8f6-4934-abc3-e7bb40703c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a55662f8acc54509b089b401af78ea9b, user_id=1d30c32e7ee441a38cdd67660fc81cd2, role_id=4f10622fcf4c4f708cd6b652d886ba3e)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:43.436843 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e544869-e8f6-4934-abc3-e7bb40703c74 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:43.573892 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1455/2904] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:40:43 2022] PUT /identity/v3/projects/a55662f8acc54509b089b401af78ea9b/users/1d30c32e7ee441a38cdd67660fc81cd2/roles/4f10622fcf4c4f708cd6b652d886ba3e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:43.578981 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cc724-ebad-46aa-88c9-2fc9ead69664 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:43.579269 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cc724-ebad-46aa-88c9-2fc9ead69664 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:43.579544 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cc724-ebad-46aa-88c9-2fc9ead69664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:43.598397 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-723cc724-ebad-46aa-88c9-2fc9ead69664 None None] MFA Rules not processed for user `7afd1ca5c60c424cb58374006df79ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:43.599587 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-723cc724-ebad-46aa-88c9-2fc9ead69664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:43.659057 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1450/2905] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:43.669503 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e544869-e8f6-4934-abc3-e7bb40703c74 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:43.675586 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3e544869-e8f6-4934-abc3-e7bb40703c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:43.728473 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e544869-e8f6-4934-abc3-e7bb40703c74 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:43.732119 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b2a37-5433-4d98-9548-f2976eeec071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:43.732752 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b2a37-5433-4d98-9548-f2976eeec071 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:43.733031 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b2a37-5433-4d98-9548-f2976eeec071 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:43.733350 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b2a37-5433-4d98-9548-f2976eeec071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:43.742252 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b2a37-5433-4d98-9548-f2976eeec071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:43.743670 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b2a37-5433-4d98-9548-f2976eeec071 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:43.752963 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1456/2906] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:40:49.126690 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9d3b7-a777-4099-a1a8-57d9d8691aa7 None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:49.127187 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9d3b7-a777-4099-a1a8-57d9d8691aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:49.127316 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9d3b7-a777-4099-a1a8-57d9d8691aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:49.155258 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-b6b9d3b7-a777-4099-a1a8-57d9d8691aa7 None None] MFA Rules not processed for user `1d30c32e7ee441a38cdd67660fc81cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:49.156558 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-b6b9d3b7-a777-4099-a1a8-57d9d8691aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:49.238254 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1451/2907] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:40:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:49.243319 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b2a37-5433-4d98-9548-f2976eeec071 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:49.253188 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b2a37-5433-4d98-9548-f2976eeec071 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:49.258460 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d15dd-278a-4ed5-9982-294c2499c090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:49.259061 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d15dd-278a-4ed5-9982-294c2499c090 admin admin] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:49.259345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d15dd-278a-4ed5-9982-294c2499c090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:49.259619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d15dd-278a-4ed5-9982-294c2499c090 admin admin] PATH_INFO: `/v3/projects` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:49.260828 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d15dd-278a-4ed5-9982-294c2499c090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:49.263054 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d15dd-278a-4ed5-9982-294c2499c090 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:49.364973 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1457/2908] 10.222.0.85 () {60 vars in 1153 bytes} [Mon Aug 29 20:40:49 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:40:49.369052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9d3b7-a777-4099-a1a8-57d9d8691aa7 None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:49.379835 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9d3b7-a777-4099-a1a8-57d9d8691aa7 None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:49.396379 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:49.436249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab25c5-55bd-4a60-a1e6-422d9f7d1470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:49.436866 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab25c5-55bd-4a60-a1e6-422d9f7d1470 admin admin] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:49.437151 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab25c5-55bd-4a60-a1e6-422d9f7d1470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:49.437427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab25c5-55bd-4a60-a1e6-422d9f7d1470 admin admin] PATH_INFO: `/v3/users` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:49.438052 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab25c5-55bd-4a60-a1e6-422d9f7d1470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:49.440000 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab25c5-55bd-4a60-a1e6-422d9f7d1470 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:49.546591 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1452/2909] 10.222.0.85 () {60 vars in 1144 bytes} [Mon Aug 29 20:40:49 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:40:49.550640 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d15dd-278a-4ed5-9982-294c2499c090 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:49.561118 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d15dd-278a-4ed5-9982-294c2499c090 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:49.565997 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee174ce9-7e0f-4432-bc9b-c63cec96c6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:49.566683 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee174ce9-7e0f-4432-bc9b-c63cec96c6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:49.567035 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee174ce9-7e0f-4432-bc9b-c63cec96c6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:49.567992 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee174ce9-7e0f-4432-bc9b-c63cec96c6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:49.568138 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee174ce9-7e0f-4432-bc9b-c63cec96c6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:49.570070 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee174ce9-7e0f-4432-bc9b-c63cec96c6e0 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:49.581853 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1458/2910] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:49.586567 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab25c5-55bd-4a60-a1e6-422d9f7d1470 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:49.596521 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab25c5-55bd-4a60-a1e6-422d9f7d1470 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:49.601917 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eb740-281e-4ef4-90cb-07b87d87b1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:49.605385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eb740-281e-4ef4-90cb-07b87d87b1f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:49.605385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eb740-281e-4ef4-90cb-07b87d87b1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:49.605385 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eb740-281e-4ef4-90cb-07b87d87b1f7 admin admin] PATH_INFO: `/v3/projects/8d45a2e20fb24ff4811f8d5ff5ab3737/users/7c9b9f066a6947b29f01e20006b1d45e/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:49.619072 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7eb740-281e-4ef4-90cb-07b87d87b1f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d45a2e20fb24ff4811f8d5ff5ab3737, user_id=7c9b9f066a6947b29f01e20006b1d45e, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:49.621777 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7eb740-281e-4ef4-90cb-07b87d87b1f7 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:49.730287 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1453/2911] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:40:49 2022] PUT /identity/v3/projects/8d45a2e20fb24ff4811f8d5ff5ab3737/users/7c9b9f066a6947b29f01e20006b1d45e/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:49.734133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee174ce9-7e0f-4432-bc9b-c63cec96c6e0 admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:49.744770 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee174ce9-7e0f-4432-bc9b-c63cec96c6e0 admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:49.748439 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:49.807598 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95ce76-3bc0-44a7-89a2-96923d1e3884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:49.808287 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ce76-3bc0-44a7-89a2-96923d1e3884 admin admin] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:49.808569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ce76-3bc0-44a7-89a2-96923d1e3884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:49.808838 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ce76-3bc0-44a7-89a2-96923d1e3884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:49.809480 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95ce76-3bc0-44a7-89a2-96923d1e3884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:49.811125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95ce76-3bc0-44a7-89a2-96923d1e3884 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:49.822577 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1459/2912] 10.222.0.85 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:49.826514 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eb740-281e-4ef4-90cb-07b87d87b1f7 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:49.837348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eb740-281e-4ef4-90cb-07b87d87b1f7 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:49.841988 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe487ee-2e7c-4813-8895-dc2679be4fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:49.842646 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe487ee-2e7c-4813-8895-dc2679be4fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:49.842926 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe487ee-2e7c-4813-8895-dc2679be4fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:49.843201 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe487ee-2e7c-4813-8895-dc2679be4fd9 admin admin] PATH_INFO: `/v3/projects/8d45a2e20fb24ff4811f8d5ff5ab3737/users/7c9b9f066a6947b29f01e20006b1d45e/roles/58f6b4d2e3664607851e63c89b5ca3d3` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:49.845143 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe487ee-2e7c-4813-8895-dc2679be4fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d45a2e20fb24ff4811f8d5ff5ab3737, user_id=7c9b9f066a6947b29f01e20006b1d45e, role_id=58f6b4d2e3664607851e63c89b5ca3d3)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:49.847787 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe487ee-2e7c-4813-8895-dc2679be4fd9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:49.858615 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1454/2913] 10.222.0.85 () {60 vars in 1483 bytes} [Mon Aug 29 20:40:49 2022] PUT /identity/v3/projects/8d45a2e20fb24ff4811f8d5ff5ab3737/users/7c9b9f066a6947b29f01e20006b1d45e/roles/58f6b4d2e3664607851e63c89b5ca3d3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:55.384134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1586c-1104-48fc-8cd3-940ba74aaf84 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:55.384134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1586c-1104-48fc-8cd3-940ba74aaf84 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:55.384134 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1586c-1104-48fc-8cd3-940ba74aaf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:55.414880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-bed1586c-1104-48fc-8cd3-940ba74aaf84 None None] MFA Rules not processed for user `7c9b9f066a6947b29f01e20006b1d45e`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:55.414880 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-bed1586c-1104-48fc-8cd3-940ba74aaf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:55.498432 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1460/2914] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:40:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:55.515205 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe487ee-2e7c-4813-8895-dc2679be4fd9 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:55.517927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7fe487ee-2e7c-4813-8895-dc2679be4fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:55.622959 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe487ee-2e7c-4813-8895-dc2679be4fd9 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:55.628943 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494fa96-3c10-41c1-8583-4496476bd7dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:55.629705 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494fa96-3c10-41c1-8583-4496476bd7dc service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:55.629994 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494fa96-3c10-41c1-8583-4496476bd7dc service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:55.630261 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494fa96-3c10-41c1-8583-4496476bd7dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:55.705209 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494fa96-3c10-41c1-8583-4496476bd7dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:55.706905 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494fa96-3c10-41c1-8583-4496476bd7dc service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:55.719588 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1455/2915] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:41:05.494779 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=60 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:05.715348 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=91,keystone:INSERT=5 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:14.221898 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16641dd-80c7-4c64-ab24-5e8ba8abd528 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:14.222425 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16641dd-80c7-4c64-ab24-5e8ba8abd528 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:14.222425 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16641dd-80c7-4c64-ab24-5e8ba8abd528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:14.242124 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-e16641dd-80c7-4c64-ab24-5e8ba8abd528 None None] MFA Rules not processed for user `1d30c32e7ee441a38cdd67660fc81cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:41:14.243374 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-e16641dd-80c7-4c64-ab24-5e8ba8abd528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:14.250214 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1461/2916] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:41:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:41:14.572457 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494fa96-3c10-41c1-8583-4496476bd7dc service nova] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:14.632157 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494fa96-3c10-41c1-8583-4496476bd7dc service nova] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:14.643571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f8e18-b388-47e8-9499-21470e779f0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:14.643571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f8e18-b388-47e8-9499-21470e779f0c service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:14.643571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f8e18-b388-47e8-9499-21470e779f0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:14.643571 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f8e18-b388-47e8-9499-21470e779f0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:14.644192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-994f8e18-b388-47e8-9499-21470e779f0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:14.649300 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f8e18-b388-47e8-9499-21470e779f0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:14.651356 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f8e18-b388-47e8-9499-21470e779f0c service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:14.659842 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1456/2917] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:41:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:41:24.236605 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=3 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:24.655781 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=26 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:35.853217 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16641dd-80c7-4c64-ab24-5e8ba8abd528 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:35.862922 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16641dd-80c7-4c64-ab24-5e8ba8abd528 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:35.866544 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab25183-68bf-4ca6-a2b2-4d3484277f47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:35.867133 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab25183-68bf-4ca6-a2b2-4d3484277f47 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:35.867413 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab25183-68bf-4ca6-a2b2-4d3484277f47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:35.867685 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab25183-68bf-4ca6-a2b2-4d3484277f47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:35.876877 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab25183-68bf-4ca6-a2b2-4d3484277f47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:35.878323 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab25183-68bf-4ca6-a2b2-4d3484277f47 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:35.888037 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1462/2918] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:41:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:41:45.883474 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=6 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:43:20.067512 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f8e18-b388-47e8-9499-21470e779f0c service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:43:20.076150 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f8e18-b388-47e8-9499-21470e779f0c service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:43:20.079799 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef169d5-d41b-4225-9667-4b019023b9df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:43:20.080975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef169d5-d41b-4225-9667-4b019023b9df service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:43:20.081253 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef169d5-d41b-4225-9667-4b019023b9df service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:43:20.081524 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef169d5-d41b-4225-9667-4b019023b9df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:43:20.083220 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3ef169d5-d41b-4225-9667-4b019023b9df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:43:20.092550 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef169d5-d41b-4225-9667-4b019023b9df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:43:20.093979 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef169d5-d41b-4225-9667-4b019023b9df service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:43:20.101802 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1457/2919] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:43:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:43:30.098247 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=6 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:43:33.444750 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab25183-68bf-4ca6-a2b2-4d3484277f47 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:43:33.454135 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab25183-68bf-4ca6-a2b2-4d3484277f47 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:43:33.458505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2616e0-915e-4129-b2f2-6a7a45f5a6e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:43:33.459143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2616e0-915e-4129-b2f2-6a7a45f5a6e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:43:33.459424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2616e0-915e-4129-b2f2-6a7a45f5a6e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:43:33.459697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2616e0-915e-4129-b2f2-6a7a45f5a6e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:43:33.461509 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-9c2616e0-915e-4129-b2f2-6a7a45f5a6e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:43:33.475246 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2616e0-915e-4129-b2f2-6a7a45f5a6e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:43:33.476716 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2616e0-915e-4129-b2f2-6a7a45f5a6e4 service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:43:33.487547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1463/2920] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:43:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:43:34.671427 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef169d5-d41b-4225-9667-4b019023b9df service neutron] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:43:34.672832 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-3ef169d5-d41b-4225-9667-4b019023b9df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:43:34.683872 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef169d5-d41b-4225-9667-4b019023b9df service neutron] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:43:34.687525 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cea5e5-3603-48a8-882f-1b52743a6c48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:43:34.688115 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cea5e5-3603-48a8-882f-1b52743a6c48 service neutron] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:43:34.688434 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cea5e5-3603-48a8-882f-1b52743a6c48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:43:34.688707 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cea5e5-3603-48a8-882f-1b52743a6c48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:43:34.691013 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-64cea5e5-3603-48a8-882f-1b52743a6c48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:43:34.703903 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cea5e5-3603-48a8-882f-1b52743a6c48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:43:34.705412 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cea5e5-3603-48a8-882f-1b52743a6c48 service neutron] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:43:34.716780 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1458/2921] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:43:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:43:36.328756 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2616e0-915e-4129-b2f2-6a7a45f5a6e4 service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:43:36.344433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2616e0-915e-4129-b2f2-6a7a45f5a6e4 service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:43:36.346550 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce4649-0b1d-4772-8711-4fffd715994f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:43:36.347150 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce4649-0b1d-4772-8711-4fffd715994f service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:43:36.347431 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce4649-0b1d-4772-8711-4fffd715994f service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:43:36.347704 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce4649-0b1d-4772-8711-4fffd715994f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:43:36.349398 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-74ce4649-0b1d-4772-8711-4fffd715994f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:43:36.361896 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce4649-0b1d-4772-8711-4fffd715994f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:43:36.363547 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce4649-0b1d-4772-8711-4fffd715994f service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:43:36.374062 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1464/2922] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:43:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:43:38.400513 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f24d4-eb50-4898-bdb1-05721355f70e None None] REQUEST_METHOD: `POST` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:43:38.400991 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f24d4-eb50-4898-bdb1-05721355f70e None None] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:43:38.401106 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f24d4-eb50-4898-bdb1-05721355f70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:43:38.420569 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.auth.core [None req-7e9f24d4-eb50-4898-bdb1-05721355f70e None None] MFA Rules not processed for user `1d30c32e7ee441a38cdd67660fc81cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81859) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:43:38.421753 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7e9f24d4-eb50-4898-bdb1-05721355f70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:43:38.428668 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1459/2923] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:43:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:43:46.369569 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=12 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:43:48.415047 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=9 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:45:37.634077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890f687-f70c-4191-81d0-a55d36db1be8 None None] REQUEST_METHOD: `POST` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:45:37.634701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890f687-f70c-4191-81d0-a55d36db1be8 None None] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:45:37.634701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890f687-f70c-4191-81d0-a55d36db1be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:45:37.664867 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.auth.core [None req-3890f687-f70c-4191-81d0-a55d36db1be8 None None] MFA Rules not processed for user `1d30c32e7ee441a38cdd67660fc81cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81858) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:45:37.666069 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3890f687-f70c-4191-81d0-a55d36db1be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:45:37.689619 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1465/2924] 10.222.0.85 () {58 vars in 958 bytes} [Mon Aug 29 20:45:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:45:38.951001 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f24d4-eb50-4898-bdb1-05721355f70e None None] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:45:38.953203 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-7e9f24d4-eb50-4898-bdb1-05721355f70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:45:39.096957 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f24d4-eb50-4898-bdb1-05721355f70e None None] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:45:39.102103 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81ec6b-7ee6-464b-9e65-95efa71bbe6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:45:39.102698 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81ec6b-7ee6-464b-9e65-95efa71bbe6c service placement] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:45:39.102976 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81ec6b-7ee6-464b-9e65-95efa71bbe6c service placement] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:45:39.103246 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81ec6b-7ee6-464b-9e65-95efa71bbe6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:45:39.104982 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-1a81ec6b-7ee6-464b-9e65-95efa71bbe6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:45:39.116111 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81ec6b-7ee6-464b-9e65-95efa71bbe6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:45:39.117693 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81ec6b-7ee6-464b-9e65-95efa71bbe6c service placement] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:45:39.127838 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1460/2925] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:45:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:45:40.514537 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890f687-f70c-4191-81d0-a55d36db1be8 None None] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:45:40.515952 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-3890f687-f70c-4191-81d0-a55d36db1be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:45:40.533428 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890f687-f70c-4191-81d0-a55d36db1be8 None None] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:45:40.540751 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d83e67-1329-4e9c-8383-0db8cae881b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:45:40.541457 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d83e67-1329-4e9c-8383-0db8cae881b6 service placement] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:45:40.541940 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d83e67-1329-4e9c-8383-0db8cae881b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:45:40.542505 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d83e67-1329-4e9c-8383-0db8cae881b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:45:40.546951 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-f2d83e67-1329-4e9c-8383-0db8cae881b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:45:40.558926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d83e67-1329-4e9c-8383-0db8cae881b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:45:40.560529 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d83e67-1329-4e9c-8383-0db8cae881b6 service placement] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:45:40.570288 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1466/2926] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:45:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:45:49.123146 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=63 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:45:50.565970 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=21 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:45:55.954610 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81ec6b-7ee6-464b-9e65-95efa71bbe6c service placement] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:45:55.967014 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81ec6b-7ee6-464b-9e65-95efa71bbe6c service placement] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:45:55.973382 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f1bfc-e771-489f-9d14-7c03931d8a8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:45:55.973975 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f1bfc-e771-489f-9d14-7c03931d8a8e service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:45:55.974255 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f1bfc-e771-489f-9d14-7c03931d8a8e service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:45:55.974527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f1bfc-e771-489f-9d14-7c03931d8a8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:45:55.976275 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-874f1bfc-e771-489f-9d14-7c03931d8a8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:45:55.988320 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f1bfc-e771-489f-9d14-7c03931d8a8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:45:55.989889 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f1bfc-e771-489f-9d14-7c03931d8a8e service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:45:56.000303 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1461/2927] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:45:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:45:56.269209 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d83e67-1329-4e9c-8383-0db8cae881b6 service placement] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:45:56.270767 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-f2d83e67-1329-4e9c-8383-0db8cae881b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:45:56.287548 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d83e67-1329-4e9c-8383-0db8cae881b6 service placement] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:45:56.293026 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d66001-5b9b-4748-ae8e-6698e67ff074 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:45:56.293676 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d66001-5b9b-4748-ae8e-6698e67ff074 service nova] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:45:56.293965 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d66001-5b9b-4748-ae8e-6698e67ff074 service nova] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:45:56.294245 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d66001-5b9b-4748-ae8e-6698e67ff074 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:45:56.303093 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d66001-5b9b-4748-ae8e-6698e67ff074 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:45:56.304748 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d66001-5b9b-4748-ae8e-6698e67ff074 service nova] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:45:56.313639 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1467/2928] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:45:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:46:01.028927 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f1bfc-e771-489f-9d14-7c03931d8a8e service nova] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:01.040294 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f1bfc-e771-489f-9d14-7c03931d8a8e service nova] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:01.045812 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa14ad1-01dd-4e5f-bd06-7ea383367cad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805df3047fee45c7a702d72a5c082eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:01.046921 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa14ad1-01dd-4e5f-bd06-7ea383367cad service nova] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:01.047204 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa14ad1-01dd-4e5f-bd06-7ea383367cad service nova] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:01.047478 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa14ad1-01dd-4e5f-bd06-7ea383367cad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:01.049273 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-afa14ad1-01dd-4e5f-bd06-7ea383367cad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:01.061655 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa14ad1-01dd-4e5f-bd06-7ea383367cad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:01.063838 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa14ad1-01dd-4e5f-bd06-7ea383367cad service nova] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:01.074669 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1462/2929] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:46:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:46:06.309608 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=6 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:11.070141 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=12 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:47:48.569009 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d66001-5b9b-4748-ae8e-6698e67ff074 service nova] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:48.589033 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d66001-5b9b-4748-ae8e-6698e67ff074 service nova] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:48.592734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68114b0b-452e-498c-99f0-57ff41afd8ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa222b1c89dc4d2cb1686b4838c7908b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:48.593326 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68114b0b-452e-498c-99f0-57ff41afd8ae service neutron] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:48.593600 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68114b0b-452e-498c-99f0-57ff41afd8ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:48.593868 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68114b0b-452e-498c-99f0-57ff41afd8ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:48.663592 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68114b0b-452e-498c-99f0-57ff41afd8ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:48.665078 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68114b0b-452e-498c-99f0-57ff41afd8ae service neutron] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:48.683208 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1468/2930] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:47:52.237243 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa14ad1-01dd-4e5f-bd06-7ea383367cad service nova] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:52.249006 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa14ad1-01dd-4e5f-bd06-7ea383367cad service nova] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:52.254046 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2671a0-ed7d-4065-a1da-37bedc95d2d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:52.261234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2671a0-ed7d-4065-a1da-37bedc95d2d9 service placement] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:52.261234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2671a0-ed7d-4065-a1da-37bedc95d2d9 service placement] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:52.261234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2671a0-ed7d-4065-a1da-37bedc95d2d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:52.261234 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-aa2671a0-ed7d-4065-a1da-37bedc95d2d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:52.269497 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2671a0-ed7d-4065-a1da-37bedc95d2d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:52.271202 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2671a0-ed7d-4065-a1da-37bedc95d2d9 service placement] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:52.282816 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1463/2931] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:47:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:47:55.965528 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68114b0b-452e-498c-99f0-57ff41afd8ae service neutron] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:55.983042 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68114b0b-452e-498c-99f0-57ff41afd8ae service neutron] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:55.989543 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8765b2-f3de-440b-a302-091a90c5d139 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:55.990170 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8765b2-f3de-440b-a302-091a90c5d139 service placement] REQUEST_METHOD: `GET` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:55.990452 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8765b2-f3de-440b-a302-091a90c5d139 service placement] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:55.990730 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8765b2-f3de-440b-a302-091a90c5d139 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:55.992697 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-7a8765b2-f3de-440b-a302-091a90c5d139 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:56.005139 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8765b2-f3de-440b-a302-091a90c5d139 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:56.006948 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8765b2-f3de-440b-a302-091a90c5d139 service placement] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:56.017926 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1469/2932] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:47:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:47:56.094006 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2671a0-ed7d-4065-a1da-37bedc95d2d9 service placement] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:56.104825 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2671a0-ed7d-4065-a1da-37bedc95d2d9 service placement] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:56.109939 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:56.114494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:56.114494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:56.114494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] PATH_INFO: `/v3/users/1d30c32e7ee441a38cdd67660fc81cd2` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:56.114494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d30c32e7ee441a38cdd67660fc81cd2)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:56.114494 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:56.354055 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d30c32e7ee441a38cdd67660fc81cd2'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:56.354667 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d30c32e7ee441a38cdd67660fc81cd2'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:56.361527 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d30c32e7ee441a38cdd67660fc81cd2'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:56.448192 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d30c32e7ee441a38cdd67660fc81cd2'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:56.448826 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d30c32e7ee441a38cdd67660fc81cd2'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:56.467116 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1464/2933] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:47:56 2022] DELETE /identity/v3/users/1d30c32e7ee441a38cdd67660fc81cd2 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:47:56.747938 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8765b2-f3de-440b-a302-091a90c5d139 service placement] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:56.758748 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8765b2-f3de-440b-a302-091a90c5d139 service placement] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:56.816079 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:56.816734 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:56.817013 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:56.817286 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] PATH_INFO: `/v3/projects/a55662f8acc54509b089b401af78ea9b` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:56.818414 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] RBAC: Authorizing `identity:delete_project(project_id=a55662f8acc54509b089b401af78ea9b)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:56.820345 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:56.923216 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a55662f8acc54509b089b401af78ea9b', 'request_id': 'req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:56.930007 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a55662f8acc54509b089b401af78ea9b', 'request_id': 'req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:56.932071 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:47:56.935701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a55662f8acc54509b089b401af78ea9b', 'request_id': 'req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:57.042626 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] The token cache is being invalidate because project a55662f8acc54509b089b401af78ea9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:47:57.043297 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:57.043607 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:57.045143 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1470/2934] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:47:56 2022] DELETE /identity/v3/projects/a55662f8acc54509b089b401af78ea9b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:47:57.049022 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:57.050440 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:57.065432 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920644f8-80e2-4b2c-86bf-e05a0be9e9a9 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:57.120700 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:57.121399 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:57.121589 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:57.121863 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] PATH_INFO: `/v3/users/7c9b9f066a6947b29f01e20006b1d45e` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:57.122978 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c9b9f066a6947b29f01e20006b1d45e)` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:57.124930 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:57.319091 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c9b9f066a6947b29f01e20006b1d45e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:57.319797 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c9b9f066a6947b29f01e20006b1d45e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:57.326592 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c9b9f066a6947b29f01e20006b1d45e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:57.330282 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:47:57.385785 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c9b9f066a6947b29f01e20006b1d45e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:57.389867 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.notifications [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c9b9f066a6947b29f01e20006b1d45e'} {{(pid=81859) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:57.410405 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1465/2935] 10.222.0.85 () {58 vars in 1224 bytes} [Mon Aug 29 20:47:57 2022] DELETE /identity/v3/users/7c9b9f066a6947b29f01e20006b1d45e => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:47:57.661856 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] Authenticating user token {{(pid=81858) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:57.663424 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.fernet_utils [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81858) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:57.678772 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f0699-a0cd-4e3a-8bb5-d0d025a604db admin admin] Validating token access rules against request {{(pid=81858) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:57.739225 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffb9f6-7253-4362-ab7a-d68ff251c969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7afd1ca5c60c424cb58374006df79ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08700a8f1f34f3488594693a51c7bb4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81858) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:57.739846 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffb9f6-7253-4362-ab7a-d68ff251c969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:57.740125 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffb9f6-7253-4362-ab7a-d68ff251c969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:57.740433 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffb9f6-7253-4362-ab7a-d68ff251c969 admin admin] PATH_INFO: `/v3/projects/8d45a2e20fb24ff4811f8d5ff5ab3737` {{(pid=81858) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:57.742175 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffb9f6-7253-4362-ab7a-d68ff251c969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d45a2e20fb24ff4811f8d5ff5ab3737)` {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:57.744081 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffb9f6-7253-4362-ab7a-d68ff251c969 admin admin] RBAC: Authorization granted {{(pid=81858) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:57.859219 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-efffb9f6-7253-4362-ab7a-d68ff251c969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d45a2e20fb24ff4811f8d5ff5ab3737', 'request_id': 'req-efffb9f6-7253-4362-ab7a-d68ff251c969'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:57.865483 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-efffb9f6-7253-4362-ab7a-d68ff251c969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d45a2e20fb24ff4811f8d5ff5ab3737', 'request_id': 'req-efffb9f6-7253-4362-ab7a-d68ff251c969'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:57.871322 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-efffb9f6-7253-4362-ab7a-d68ff251c969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d45a2e20fb24ff4811f8d5ff5ab3737', 'request_id': 'req-efffb9f6-7253-4362-ab7a-d68ff251c969'} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:57.968806 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-efffb9f6-7253-4362-ab7a-d68ff251c969 admin admin] The token cache is being invalidate because project 8d45a2e20fb24ff4811f8d5ff5ab3737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81858) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:47:57.969077 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-efffb9f6-7253-4362-ab7a-d68ff251c969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:57.969701 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG keystone.notifications [None req-efffb9f6-7253-4362-ab7a-d68ff251c969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81858) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:57.971189 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: [pid: 81858|app: 0|req: 1471/2936] 10.222.0.85 () {58 vars in 1233 bytes} [Mon Aug 29 20:47:57 2022] DELETE /identity/v3/projects/8d45a2e20fb24ff4811f8d5ff5ab3737 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:48:07.528392 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:48:07.965493 dv-cinder-852009-5-iscsi devstack@keystone.service[81858]: DEBUG dbcounter [-] [81858] Writing DB stats keystone:SELECT=60,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81858) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:51:22.738543 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] Authenticating user token {{(pid=81859) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:51:22.739952 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:51:22.808255 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30e983-bf45-4ef7-ae20-02047b8452f5 admin admin] Validating token access rules against request {{(pid=81859) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:51:22.867339 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4d456-0f82-41ea-9891-8cab73e16c94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5288af2dbc15438c8b209b0ba2454cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11372fe8994543b4a19773c0e0bee53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81859) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:51:22.867942 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4d456-0f82-41ea-9891-8cab73e16c94 service placement] REQUEST_METHOD: `GET` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:51:22.868249 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4d456-0f82-41ea-9891-8cab73e16c94 service placement] SCRIPT_NAME: `/identity` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:51:22.868523 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4d456-0f82-41ea-9891-8cab73e16c94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81859) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:51:22.870223 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.fernet_utils [None req-34b4d456-0f82-41ea-9891-8cab73e16c94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81859) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:51:22.881647 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4d456-0f82-41ea-9891-8cab73e16c94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:51:22.883225 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4d456-0f82-41ea-9891-8cab73e16c94 service placement] RBAC: Authorization granted {{(pid=81859) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:51:22.892913 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: [pid: 81859|app: 0|req: 1466/2937] 10.222.0.85 () {60 vars in 1365 bytes} [Mon Aug 29 20:51:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:51:32.888720 dv-cinder-852009-5-iscsi devstack@keystone.service[81859]: DEBUG dbcounter [-] [81859] Writing DB stats keystone:SELECT=58 {{(pid=81859) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}